The title is my main question but I came up with this question after watching a video from Vox showing the empires that existed in the Middle East. I'm aware that the information in the video could be incorrect but its the idea of Saudi Arabia's land being untouchable.
Link to the video: https://www.youtube.com/watch?time_continue=135&v=F4U0SXz2DJs
Saudia Arabia is, minimally, the combination of three rather different cultural zones: the Hejaz region on the West Coast which contains the Muslim holy cities of Mecca and Medina, the Najd desert and oasis region in the center, the historically mostly Shi'a East Coast along the Persian Gulf.
The desert center was difficult to govern. Indeed, the Ottomans treated it some ways like Europeans treated their colonies, as Selim Deringil argues in a lot of his recent work ('"They Live in a State of Nomadism and Savagery”: The Late Ottoman Empire and the Post-Colonial Debate' is a typical example). Here the nomad was "anathema" to modernity, and the Nejd was almost entirely nomadic while the Hejaz and the Persian Gulf side were (relatively) settled. To add tot that, the Ottomans effectively controlled the coasts, more or less, from the 16th centuries until WW1, but only intermittently had control over the center. Especially after the First Saudi State in the early 19th century, they associated the region with "religious fanaticism" which explicitly challenged their rule.
The Ottomans made extensive improvements to the era, especially notably with Hajj caravans from both Cairo and Istanbul which distributed wealth to the Hejaz, and culiminating with the famous nut ultimately unfinished Hejaz Railway which sought to bring industrial convenience and Ottoman modernity to the region. It was part of one the few railway in the Ottoman Empire east of the Bosporus, and sought to connect Istanbul to Mecca, though only got as far as Medina before work was interrupted by World War I. The Hejaz line and the ambitious "Berlin to Baghdad railroad", built simultaneously, were the first Ottoman railroad built in Arab majority regions (there had previously been railroads in the Ottoman Balkans and Anatolia, as well as British Egypt).
Control of Mecca was incredibly important in the Muslim World—it was one of the main ways dynasties established legitimacy. We saw the first Muslim Empire (the Rashidun Caliphate) come out of the Hejaz region, then the area was controlled by the Umayyads, then the Abbasids, then the Mamluks (who kept the Abbasids as nominal caliphs), then the Ottomans, then as an independent state before being swallowed up by the Saudis coming out of the Nejd.
The video, with its mostly Europe+Jerusalem focus, doesn't include the Ummayids, the Abbasids, or the Mamluks. It does include the Seljuks, who briefly controlled parts of the Hejaz (but I think neither Mecca nor Median—it's not an area of history I know well, though). So the simplest answer is that, after the Roman Empire, most of the region was controlled by a major empire, just not ones featured in the video.
As for why no one conquered it before, well, some people did. Small parts of the East Coast of the current Saudia Arabia were conquered at various times by the Romans and the Byzantines. A Roman expert can give more details on this, but essentially all the goods they wanted came up the coast and across the desert to them, they had little need to conquer the area. Generally, the Romans favored rich agricultural areas and were less concerned with deserts and such, for obvious reasons.
The Persian Empires, the Roman/Byztanine's great rival in the Middle East, often had more influence in Arabia, but this was generally on the eastern, Persian Gulf coast of Arabia, rather than the Hejaz. Indeed, subsequent Persian Empires also controlled this region intermittently. After 1501-ish, these empires were Shi'a, and the historical Shi'a majority along Saudi Arabia's east cosr is a testament to this Persian influence (I say historical majority because it's unclear the population dynamics in present Saudi Arabia; most of the Kingdom's oil is in this Shi'a region and they have conducted population transfers to try and minimize the potential for rebellion here).
Mostly, though, it's a desert region with few resources and many nomads who are adept at fighting and fleeing and who would prefer not to be taxed. Trade routes for certain goods (at various times, spices, perfumes, coffee, and leather) went through the region, but before Islam made the Hejaz symbolically powerfully, it was rarely worth an empire's effort to conquer and administer the region. After that, the coasts (especially the Hejaz) were mostly controlled by one great empire or other, but these Empires just don't figure into that map.
The most fertile parts of the Middle East since antiquity have been the areas around the Tigris and Euphrates, the Mediterranean Coast, along the Nile (these there making up the Fertile Crescent), plus the Anatolian and Persia. Which basically leaves out Arabia as a specifically infertile region, comparatively, and I think goes a long way towards answering the question about why it was a relatively low priority for regional rulers. When they did conquer the area, they mainly conquered one or both of the coasts, with the even more arid interior just not being worth the effort.