There was a great reply by mod u/celebreth a few months ago about exactly this: https://www.reddit.com/r/AskHistorians/comments/bpwo6d/did_rome_ever_try_to_push_beyond_the_nabataean

In 26 B.C.E, the emperor Augustus ordered a mission of exploration, diplomacy and potentially conquest deep into the southern part of the Arabian peninsula--what we now know as Yemen, called by the Romans "Arabia Felix" or "fertile Arabia" owing to its arability relative to the rest of the peninsula. The expedition was led by Aelius Gallus, Praefect of Egypt and friend of the famous geographer Strabo. Arabia Felix, owing to its central location on the shoreline of the Indian Ocean, had something of a monopoly on the import of cinnamon and other fine spices from Africa and Shunga/Satavahana India, and ultimately the Romans desired to either befriend or subjugate its rulers. Trade between Roman Egypt and India had been extensive in the 2 decades since Egypt's conquest; Roman merchants regularly stopped over at the port of Aden, so the riches in the area were well known.

The expedition failed miserably. Gallus and his forces fell victim to the machinations of a certain Arab named Syllaeus. This man, a guide provided by the rulers of Nabatea (modern north Arabia/southern Levant) led the Romans in circles through the unforgiving wastelands of the central peninsula, and not long after a violent disease began to ravage their ranks. By the time they reached the capital city of Ma'rib, the Roman forces had been decimated by illness and Gallus's halfhearted and fruitless siege lasted only a week, although his fleet managed to reach and destroy the port of Aden, securing Rome's dominance vis a vis trade with India. He left, returning to Alexandria with a fraction of his original force.

Fast forward 130 years, to the reign of Trajan. The king of the Nabateans--who had operated as a client kingdom and frontier buffer state ever since Rome's arrival in the Levant--died. Although this king did have a designated heir, Trajan saw this as an opportunity to expand his empire yet further, and he did so. Roman forces moved into the area relatively unopposed, and Nabatea was integrated into the empire as "Arabia Petrea".

Although not especially remarkable in terms of material productivity, Arabia Petrea became something of a hub of Roman culture in the east, to the point that Arabic apparently ceased to be spoken in the province in favor of Greek and Latin. The province demonstrated deep loyalty to the central empire on at least two occasions: first, when the Syrian general Avidius Cassius attempted to usurp an ostensibly mortally ill Marcus Aurelius in 175 AD, and again 20 years later when the governor of Syria, Pescennius Niger, declared himself emperor during the Year of the Five emperors. Neither claimant received economic or military support from Arabia Petrea. When Septimius Severus emerged victorious from the chaos, the formerly rebellious provinces of Syria and Egypt were stripped of some of their status while the opposite was true for the loyal Petreans.

Later, as the western empire began to collapse in the 4th and 5th centuries, the Byzantines divided the province into various smaller administrative units, namely Palaestine Salutaris. It became a frontier in the ever-raging war between Constantinople and the Sassanid Persians. In the very early days of the Islamic conquests--possibly beginning under Muhammad himself, but certainly completed by his successor Abu Bakr-- the area was taken from the Byzantines and incorporated into the Rashidun Caliphate.