I've heard that the people who became early Muslims were mostly believers in the now-extinct Arabic pagan religion. But were a significant number of them Jewish? What about before the advent of Christianity - what was Arabic Jewry like then?
Most? Very doubtful. A significant number? Absolutely. We of course don't census numbers on such things, but we do have accounts of Jewish tribes in various parts of Arabia as well of the Himyarite Jewish Kingdom of Yemen (although as I understand it the extent to which they were straightforwardly Jewish as opposed to merely monotheistic is a matter of dispute). That conversion of the kings of Himyar was in the 4th century AD, which is around the time when northern Arabia starts to look increasingly Christian as well.
In terms of pre-Christian Arabic Jewry, we're probably talking about a much smaller although quite ancient community in northern Arabia. So we have an inscription dated to 42 AD in Hegra (modern north-west KSA) of a tomb where the writer describes himself as a Jew. There were Jews in Southern Arabia prior to the conversion of the Himyarite kings themselves though it's my understanding that we don't know quite how old that community is.
Source wise and for further reading see Robert Hoyland's Arabia and the Arabs: From the Bronze Age to the Coming of Islam.