Yemen's recent history is a tale of division and war, which makes it hard for the country to grow the kind of sustained economy you see from other places on the Arabian peninsula or around the world.
The Sarawat mountains run along Yemen's western coast and many cities are located there, including the capital San'a. This kind of terrain can make governing difficult. Transportation and communication become more difficult, particularly prior to the last few decades of technological developments in these areas. A more fractured society is an almost inevitable result.
The mountains, along with the climate (frequent droughts), also create a situation where agriculture becomes difficult. According to the World Bank, only 2.4 percent of Yemen's land area can be farmed. While that's a higher percentage than other Arab states, it's still low, and Yemen is also working with a much smaller land area than many other nations in the region.
Furthermore Yemen is, comparatively to other Gulf states, not a rich source of oil, as it has smaller fields that were discovered much later than its counterparts. It does not belong to OPEC.
Nevertheless, agriculture and oil production make up a significant part of Yemen's economy.
It's intrinsically difficult hard for a country where communication and transportation are challenging and the leading industries are so constrained by a lack of natural resources to be economically prosperous.
Like other Arab states, much of Yemen gained independence after the collapse of the Ottoman empire in 1918.
The Mutawakkilite Kingdom of Yemen existed from 1918 to 1962, but to say they ruled over all of what you would consider modern Yemen would be an overstatement.
First, the aforementioned geography made governing a difficult task.
But perhaps more notably, the British had maintained the Aden Protectorate, in the south of Yemen, since 1874. While the Ottomans had mostly accepted this arrangement, the Mutawakkilite Kingdom dreamed of uniting the territory. This desire led to decades-long fighting along and over the border between the two entities. Beyond the border skirmishes, the Mutawakkilite Kingdom also did what they could to support tribal revolts within the protectorate.
The Mutawakkilite Kingdom faced its own share of in-fighting, too. The first imam was assassinated in a coup attempt in 1948, effectively deposing the regime until his son succeeded him as imam months later. There was another coup attempt in 1955, and then finally a successful one in 1962 after a power vacuum that followed the death of the second imam.
All of these pressures made it difficult for Yemen to place the same emphasis on modernization that other Arab states exhibited during this time.
After years of struggle to maintain control, the British finally departed for good in 1967.
Yemen became divided into two countries, with effectively a military dictatorship (Yemen Arab Republic) in the north and a socialist satellite state of the Soviet Union (People's Democratic Republic of Yemen) in the south, from 1967 through 1990.
Both nations dealt with infighting (including civil wars in the South) as well as tension with one another.
Neither state did well economically, due in large part to the challenges mentioned above.
Yemen unified in 1990 to form the Republic of Yemen.
Things might have turned around for Yemen economically here, but the Iraqi invasion of Kuwait threw a wrench into things. Yemen opposed other (non-Arab) countries intervening, essentially allying itself with Iraq. Saudi Arabia responded by kicking nearly one million Yemeni workers out of the country and reducing economic aid to Yemen.
As we've seen from other divided countries who unified, integration is not a straightforward process. The Persian Gulf War created an economic crisis in the Gulf at a time when the newly unified Yemen could have used a lifeline. And by choosing Iraq, Yemen put itself on the losing side. It lost out on a lot of aid that could have helped.
The first Yemeni Civil War since unification happened in 1994 and the current one began in 2015. Wars are, in so many ways, economically expensive. It's hard to produce oil and involve yourself in the world economy when your country is at war.
Geographic considerations alone render it virtually impossible for Yemen's economy to resemble Saudi Arabia's or Iraq's.
Combined those limitations with decades of division and war, and you have a struggling country that doesn't have the resources, infrastructure, or peacetime needed to develop.