This is primarily based on David W Anthony's work, particularly The Horse, the Wheel, and Language, though I will also draw note to Philip Kohl's critical (though ultimately laudatory) review "Perils of Carts before Horses". JP Mallory's In Search of the Indo-Europeans may also be of some use, as it also discusses some of the problems with combining archaeology and historical linguistics and it is fairly accessible, but the book is 30 years old now and is simply out of date. Christopher Beckwith's Empires of the Silk Road may also be of note as a broad history of the Near East, but I'm afraid I can't speak to its merits in terms of scholarship, as that's not my field (I have my own issues with it, and it's not in the subreddit's booklist, so it's probably better to pick one of those).

To put it briefly, Indo-European languages aren't found in the Arabian peninsula for much the same reason they aren't found among the Indigenous Peoples of the Americas: Indo-Europeans, so far as we can tell, simply didn't live there. Or, at least not at the mass required to be able to displace and/or assimilate (be it through hostile or peaceful means) the contemporary peoples of Late Copper Age Mesopotamia and the Levant.

Due to the nature of counterfactuals like this, it's impossible to say why Indo-Europeans could do one thing and not another. We don't even really know why Indo-Europeans were able to proliferate where they did, let alone not proliferate where they didn't. Researching this kind of thing is especially difficult because prehistoric languages don't leave material evidence. There are definitely things we can infer due to stylistic continuity or geographic association (e.g., if we follow the migrations of a style of pottery over the course of a couple thousand years and by the end of it, that pottery is found among definitively Indo-Aryan artefacts, we can infer that Indo-Aryan speakers may have migrated along with that style of pottery), but ultimately, nothing is certain.

The current most accepted origin theory of the Indo-European people group is that they were a nomadic steppe people (somewhat akin to Classical Turkic and Mongolic peoples) from the Pontic steppes, the grassland plains just north of the Black Sea in modern-day southern Russia. Sometimes this Urheimat is connected directly with the Yamnaya culture, but this is just a step too far for many people's, including my own, liking (i.e., it's very likely that the Yamnaya culture spoke an archaic Indo-European language, but whether that language was Proto-Indo-European and whether it was descendents of the Yamnaya who migrated is very much open for debate). From there, they migrated west and east, rounding the Black and Caspian Seas, respectively, in a couple waves (very likely no fewer than two, but upper bounds and specifics are controversial), and then proliferated from there.

Why east and west and not north and south? We don't know. To the south are the Caucasus Mountains, which would pose a problem for a horse-based people; I think it's safe to assume that that was a sufficient block, even if they were aware of the bountiful Fertile Crescent on the other side (though consider, to critique one of your assumptions, why a nomadic people would find seasonal agricultural floodplains valuable in the first place, even if they did know about them).

One of Anthony's core arguments is that, because Indo-Europeans have common words for things like wheels (e.g., wheel in English, kyklos in Greek, cakrá in Sanskrit) and woollen textiles (e.g., wool in English, lênos in Greek, ū́rṇā in Sanskrit, ḫulanaš in Hittite), and because neither of those predate about 4000 BCE, Indo-European must have been a single spoken language (or, more likely, a mutually intelligible dialect continuum with lots of internal migration) until then, maybe even until as late as 3000 BCE.

Anatolian languages, among them Hittite, are both the earliest attested Indo-European languages (we find Hittite personal names in 1900s BCE Old Assyrian texts) and show the largest lexical divergence from that common Indo-European ancestor (you will notice that there may not be an Indo-European-derived Hittite word for "wheel"), so it is generally accepted that they were the first to migrate out of the Indo-European homeland, with an Archaic Indo-European group moving into the Danube basin around 4200 BCE, and a Pre-Anatolian group moving into the Balkans, then crossing the Bosporus into Anatolia sometime around 2800 BCE. From there, groups like the Luwians and Hittites moved east into the Fertile Crescent by 2000 BCE, and partook in the administrative and agricultural advancements of their already established Near Eastern neighbours. Akkadian, a Semitic language, may be attested in Sumerian texts as early as 2800 BCE; Sumerian, an isolate clearly, it's related to basque, chinese, and navajo, is definitely attested to 2900 BCE and proto-writing in Sumerian may be attested as early as 3500 BCE.

The other group of note here are the Indo-Iranians, who emerged from a later migration out of the eastern end of Late Proto-Indo-European continuum (generally connected to be the Sintashta culture) sometime after 2500 BCE, when they rounded the east of the Caspian and ended up in the Central Asian steppes (modern-day "stanistan"; this is generally connected to the Andronovo culture) and were influenced by the Bactria–Margiana Archaeological Complex cultural group. From there, no fewer than four waves of migration (may have) occurred, two(-ish) of which are important to us:

1. The earliest migrants out of the Indo-Iranian Urheimat seem to be a group that migrated all the way out west to the Fertile Crescent, forming the elite class of the Mitanni state sometime around 1600 BCE. Even though the administrative and popular language of the Mitanni state was Hurrian, which isn't Indo-European, many elites and deities have what look to be Indo-Iranian names that are close enough to Vedic Sanskrit (e.g., the Mitanni name bi-ir-ya-ma-aš-da and the Indo-Aryan Priyamazdha) that it would be unlikely that this group didn't share a recent common origin with Indo-Aryans. The Kassites, another non-Indo-European group, show up in eastern Mesopotamia around 1600 BCE and some of them also have Indo-Iranian names and deities, but it's not so widespread as with the Mitanni.
2. The Iranians continued to round the Caspian and migrated westward into the Iranian plateau nearly a millennium later, with the Medes, Parthians, and Persians ending up in the eastern part of the Fertile Crescent around 800 BCE.
3. The Armenians are (maybe) not Indo-Iranians, but I will include them here. They either rounded the Black Sea and crossed the Bosporus (maybe they're the Phrygians?) from Europe, rounded the Caspian Sea with the Indo-Iranians (maybe it was part of a Graeco-Armenio-Iranian continuum?), or maybe they just crossed the Caucasus by themselves. Either way, they end up in the Armenian highlands by 200 CE (not enough study has been done on Armenian migration; the Armenian mythical founding is in 2500 BCE, but this is highly unlikely to be true). Pre-Armenian may have been spoken alongside Urartian (related to Hurrian) in the 800s to 600 BCE, but we ultimately don't know.

So why didn't these Indo-European groups spread throughout the Fertile Crescent and Arabia like they had in Europe or Northern India? Well, they did. Hittites had numerous wars with their Bronze Age neighbours, the Mitanni controlled almost all of modern-day Syria in 1400 BCE, the Achaemenids conquered all of the Fertile Crescent in the 500s BCE, Alexander the Great conquered the Achaemenids in 330 BCE, the Parthians and Sasanians would conquer it back. Sometimes Hittites, Persians, and Greeks were successful in establishing their language in administrative (and even popular) use, other times they weren't. The same holds true for languages like Kassite and Assyrian and Arabic.