You’re right. We know remarkably little about pre-Islamic Arabia compared to a lot of surrounding areas in that period and there’s several reasons for that.
Now, to be fair, Arabia is surrounded by regions that are very well recorded. Egypt is consistently one of the best evidenced regions throughout human history, (1) as is Mesopotamia, as is Palestine. Given that the Byzantine/Romans control large areas of the region and that the Sassanian Persians much of the rest on the eve of Islam’s rise, it’s perhaps not surprising that we have comparatively more records for these neighbouring areas.
That brushes upon one of the first reasons we don’t know much about pre-Islamic Arabia: there were very few groups, societies, or polities that left much for us to find. I don’t want to give the wrong impression here: there were territorial states in Arabia before Islam such as the Himyarites in Yemen, but they were few and far between. There weren’t any big bureaucratic entities like Rome or Persia that left behind the records, buildings, inscriptions, coins, and more for us to find in the first place. Nomadic groups, of which Arabia had many, are infamously difficult to find in historical records, while the major cities and towns were few.
This limit in source material is expressed in literary and material terms. We do not have clear pre-Islamic historians from and focusing on the Arabian peninsula. Combined with such limited material sources, we simply do not have the evidence to know as much as we could for so many other regions.
There were, of course, other writers outside of Arabia, but at best they deal with Arabia offhandedly, if at all. Pre-Islamic Arabia only really gets talked about in these sources to the extent that it relates to the interests of the major nearby states. Procopius discusses Arabian land only in relation to what Emperor Justinian holds there, for example. (2) It is only the Roman or Persian allies in Northern Arabia that get much attention, the Lakhmids and the Ghassanids, and even then only when it’s relevant to the major states, such as their inclusion in treaties between the states. (3) There is mention of the Aksumite invasion of Himyar (Yemen) in the 6th century, (4) but again, it’s only when a major external state steps in that it becomes worthy of comment. This lack of focus again leaves us with much less to go on that these writers left for other regions.
It’s worth mentioning that Arabia’s environmental conditions don’t do much to preserve some evidence either. This might be more relevant to very early Islamic Arabia, though I suppose it’d be relevant to pre-Islamic Arabia as well, but things like papyri don’t survive well in Arabia’s climate. (5) It’s frankly just too warm and dry a lot of the time. Considering how much papyrological evidence we have for Egypt in this time by comparison, it’s fair to say that even if there had once been evidence, it’s unlikely such sources would have survived through to us now.
As for archaeology, there’s been relatively little of it performed compared to other regions. One more delicate issue is the issue of Islamic convention and tradition. Islamic tradition holds quite a clear impression of pre-Islamic Arabia as a region of disconnected pagan nomads that, historically speaking, probably isn’t quite accurate. (6) The idea of contravening the established tradition is controversial so it’s not a topic area many people, particularly in the Arabian peninsula itself, are eager to do. There’s also the issue that the few major population centres we do know of in pre-Islamic Arabia - namely Mecca and Medina - are, for obvious religious reasons, not somewhere that is easily excavated. A reluctance or inability to study it then might partially explain it.
So overall it’s a combination of a lack of evidence in the first place, difficult conditions for preserving potential evidence, and a reluctance or inability to dig too much into Arabia’s past before Islam.
Footnotes
(1) - P. Crone & M. Hinds, God's Caliph : Religious Authority in the First Centuries of Islam, (Cambridge, 1986), p287
(2) - Procopius, The Wars, 1.19
(3) - Menander Protector, History of Menander the Guardsman, 6.1
(4) - Procopius, The Wars, 1.20
(5) - P. Crone & M. Hinds, God's Caliph : Religious Authority in the First Centuries of Islam, (Cambridge, 1986), p293
(6) - H. Munt et al, ‘Arabic and Persian Sources for Pre-Islamic Arabia’, in G. Fisher (ed.), Arabs and Empire Before Islam, (Oxford, 2015), p435