The Arabian peninsula is extremely large, and it's hostile terrain. The desert in the center was all but impassable without massive preparation and the assistance of natives. Mountains and hills along the western coast slowed marches and provided ample opportunities for ambush.

Rome under Augustus mounted an expedition down the western coast, which ended in utter disaster. (For more on the expedition of Aelius Gallus, I would defer to an expert.) Roman probes into Nubia were also repelled.

Roman decisions about conquest and war were not always based on a cost-benefit strategy we would recognize, and their definition of realpolitik was different from ours (see Susan Mattern's Rome and the Enemy: Imperial Strategy in the Principate), but basically, since there wasn't a lot of profit or prestige at stake, Rome didn't have a pressing reason to continue fighting in Arabia. The local kingdoms were content to have a stable trading partner in Egypt and the Levant, and the Ghassanid Arabs maintained physical and diplomatic buffer room.

Correct, the Arabian peninsula is not that lucrative of a land grab. Partly because many of its peoples were somewhat nomadic herdsmen, but largely because there weren't that many of them to begin with. To give a sense of the region's current population density, imagine if the entire earth was one land mass, but contained no additional people. That's the same population density that Saudi Arabia currently has. Oman is actually less dense.

Tidbits: Interactions that did happen focused on the most populated region of the western Red Sea coast. Yemen was known as "Punt" to ancient Egpytian- Hatshepsut's temple at Deir-al-Bahri boasts about successful trade relations with queen there, shown as a plump woman laden with plates of spices and incense. Hatshepsut's architect Senmut even placed trees from Punt in the temple courts (Richard Wilikinson, Complete Temples of Ancient Egypt). Rome did occasionally attempt to enter the region for the same goods, but with little success. Eventually, this region intermingled with the customs and language of Ethiopia, to which it is closer than, say, the Levant.

One argument that I've read had to do with the relative position of Arabia in relation to the trade routes - namely, not that important. Prior to the completion of the Suez canal, the trade routes were either through Palestine/Mesopotamia, or entirely by sea around Africa.

With the creation of the canal, the trade route started to skirt the Arabian coast, making it a valuable diplomatic territory. Add on the conversion to steam power, and you now have a nation like Britain being very concerned with the route through the Red Sea, and with a sudden need for coaling stations. Yemen, if I recall correctly, is the result of British India colonizing Aden and setting up a coaling station. India was also instrumental in setting up the British treaty system with the Gulf States to ensure that no foreign power would gain a foothold near the Empire's vital sea route.

Prior to this, the Ottoman Empire's interest was simply in controlling Mecca in support of the Caliphate. To that end they built a railroad in Hejaz to control the pilgrimage to the holy city, but largely ignored the territory otherwise. Nedj, where the Saudis got their start, was so barren as to be useless to empire. Nedj was largely a feudal oasis society that used Bedouins as mercenaries until Ibn Saud popularized Wahhabism and turned the mercenaries into religious warriors.

If it wasn't for WWI, Saudi Arabia might have been unified earlier. The British paid Ibn Saud to remain neutral, while they courted the authorities of Hejaz to rebel against Ottoman rule. The Indians had suggested that Saud was the better horse to back, but the authorities in Egypt were more familiar with Hejaz and thus history is written. And, of course, by that point the oil potential of the future mandate territories was known, so the British made sure that their allies from Hejaz inherited the thrones of Jordan and Iraq.

But even then, when the British were the masters of the Middle East, they didn't bother with Arabia proper. They essentially created a fence around Ibn Saud, and allowed him to unify the portions of the peninsula that they deemed unimportant (including Hejaz). It was only later after they laughed at the Americans for drilling for oil in Arabia that they realized that there might have been serious strategic value in courting Ibn Saud.

So to add to what's been said from the opposite perspective, of Arabs moving out of Arabia, one of the things I haven't seen mentioned is the difference between government and administration of lands based around trade or nomadic agriculture (granted, with cities as the focal point of both) and settled communities. The Arab conquests caused huge dissension culminating in civil war at the root of which seemed to be much consternation over whether the Arabs were to be nomadic tribesmen like their forefathers with all of the virtues and freedoms that accompany that or rulers of settled peoples. And in their case that shift from one to the other is matched by the movement of the capital from Medina out of Arabia, most notably to Damascus.

It's worth noting that by the late antique period immediately before the Arab conquests, both the Byzantines and the Sassanians, obviously very close in proximity to Arabia, both chose to maintain hegemony over the region through client kingdoms of tribal confederations rather than direct administration.

As for an interesting tidbit, I've been reading the Sira lately, which includes some background on the tribes of Arabia before Muhammad (granted the Sira was written hundreds of years later, has historiographical problems etc. etc.) but was amused to discover that the phrase used for "settled communities" translates literally as "Mud people." So you have the "noble tribes" or the "highland tribes" and the "mud people." Presumably referring to their building materials.

Sourcewise: On the dissent of the tribes Watt's "Formative Period in Islamic Thought" covers the first fitna (civil war.) The Ghassanid and Lakhmid client states will be covered in any decent history of the era or of the Byzantine-Sassanian wars, but the entry in "Late Antiquity: A Guide to the Post-Classical World" is probably enough to confirm what I've written even at a couple pages.

Most of central Arabia is inhospitable desert. Note, however, that north Arabia and the southern costal regions around Yemen were very important throughout antiquity. The Babylonian king Nabonidus, for example, erected a stela at the Tayma oasis and the Byzantine Empire, Axum and Sassanid Persia all intervened at various points in the various South Arabian polities, especially the ones in Yemen. A good introduction to ancient Arabia might be the exhibition catalog Roads of Arabia if you can tolerate the more irritating bits of Saudi propaganda. It may also be worthwhile to take a look at Stuart Munro-Hay's Aksum: An African Civilization of Late Antiquity because although it is not in ancient Arabia proper Axum had longstanding close trade ties with Arabia and this is reflected in Axumite material culture.