My time to shine, I wrote my BA thesis on the linguistic landscape of Yemen between Late Antiquity and the early Islamic Period. That being said, I'm not really sure whether to interpret your question as pertaining to the cultural or linguistic arabization, but I can talk a bit about both.

So what happened in Yemen from about the first century CE onwards is that one of the tribes previously under Saba'ic control thought it would be a great idea to become more independent: they formed an alliance with some other tribes and formed a confederacy and became known as the Himyarites. Throughout the following centuries these Himyarites decided it would also be cool to just rule the entirety of Yemen alone, and so they swallowed up all of the surrounding kingdoms in the area throughout the following centuries. Their expansion had gained the attention of the two main superpowers in the area, particularly due to the fact that the Himyarites were now able to control the majority of trade through the Strait of Aden (nowadays mostly known for Somali pirates), and as a result, both empires sought to influence the Himyari court; particularly through religion. Although there are some indications that the Himyarites were in the beginning somewhat sympathetic to the Byzantine Empire, in the end they threw their in their lot with the Persians instead.
That turned out to be a bad idea, as Yemen was promptly invaded by the Byzantines' Ethiopian allies, and after several attempts at regaining their independence had failed, Ethiopia decided to start assuming direct control of the area, signaling the end of the South Arabian kingdoms that had existed for over a millennium.
So how do we know all this? Basically, our most important source of information comes from a ridiculous amount of inscriptions (ranging in the tens of thousands at least), which range from very formal monumental and religious inscriptions incised by professional masons on the order of their kings and trading writs written on palm leafs up to very personal graffiti carved into rocks by bored shepherds. In any case, by the time the Ethiopians invaded, which is in the year 520 CE, we basically find a few last inscriptions. Afterwards, local writing tradition ends.

As to what happened afterwards? We don't really know. As Rex-Smith puts it: "one of the greatest frustrations […] is the plain fact there is so little information available concerning what can be called the “pre-dynasty” history of the country". The Islamic tradition doesn't really provide us with much helpful information either: there are some ḥadīth mentioning how Muhammad sent a letter to a guy called Bazan, the governor of San‘ā, who, along with the entirety of his populace, was supposed to have converted to Islam on the spot, but the fact that Yemen was no longer under Persian control (and hadn't been for some time) indicates that this recounting is likely apocryphal. There is also a cool story from al-Tabarī's Histories, in which he recounts how the Himyari king Yu’sar Yu’far al-An‘am (note that he actually existed!) left an inscription on the edge of the desert warning everyone of certain death. Tabarī provides its supposed text, but the fact that its written in Classical Arabic makes it unlikely that it was composed by this Himyari king. What is however significant is the fact that this demonstrates that the Arab historians had a pretty good understanding of the history of Yemen in general.

So then what do we know? The Islamic tradition does indicate that many "Yemeni" tribes aided in the earliest Islamic conquests, but we're not sure whether they mean Arabic tribes living in the Greater Yemen area or whether they actually meant Ancient South Arabian-speaking tribes. It's cool that al-Baladhūri talks about the conquest of Damascus and provides us with the name of the general as Shurahbil, which is basically as South Arabian as John is English (i.e. very common). At the very least there was at least at one Yemeni guy there, if not more. There is also a book by Fred Donner about the Arab tribes participating in the conquest of Iraq, and it's possible that there are Yemeni tribes mentioned there as well.

As for the linguistic evidence, it is both exciting and a bit paradoxical. Firstly, it must be said that the Arabic dialects of Yemen are amongst some of the most conservative in the Arabic-speaking world. I don't have to go into detail, but suffice to say that they contain many of the features also found in the oldest attested varieties of Arabic. It's odd, though, that the majority of these strongly conservative dialects are urban dialects, although generally these are the ones to change the quickest. The dialects of the surrounding rural areas are generally less conservative. However, a very interesting feature of some Yemeni dialects is the fact their definite article is not al-, as in most other varieties of Arabic, but am- instead. Whether this features is a remnant of South Arabian influence or a later Arabic innovation is a matter of some controversy, but it's interesting to notice that in Late Sabaic, the definite article was added to the end of a word (like in Swedish or Romanian) and was probably pronounced -am as well. What's really cool is that this is supported both the Islamic hagiographical tradition (there is a story in which the Prophet Muhammad is supposed to have said laysa am-birr am-ṣawm fī am-safar, 'there is no compulsion to fast while traveling), as well as by later Muslim travellers to the area.
One of these travellers, the 10th century al-Hamdāni, who was actually a native of the area, but who had spent the majority of his life in Iraq, went down to the area and described the speech of the people he came into contact with. Basically Hamdani judged these different speech forms on a level from "basically OK Arabic" to "completely incomprehensible gibberish". What is important to note, however, is that Hamdani was not a modern dialectologist interested in objectively describing the various linguistic features of a dialect, but he was trying to match what he heard with his ideal version of Arabic, meaning the Classical Arabic at the court in Baghdad. As such, it's questionable to which degree we can use Hamdani to make any statements about the degree to which the Yemen had been linguistically Arabized.

I'll end this post with something else. Basically, when the early Arabic philologers used the word Himyari, they used it as a kind of shorthand to mean "weird and old Yemen". This is emphasized by a famous anecdote in which an Arab envoy was invited to the court of a Yemeni king. The moment the the envoy entered the room, the Yemeni king said to him thib, and the envoy promptly jumped out of the window. The joke comes from the fact that in Arabic, thib means "jump"; in the Himyari language it's supposed to mean "sit" (as it does in Hebrew AFAIK). The point is that the Arab writers were very much aware of the fact that Yemen could be foreign and ancient, and in some part in continued to hold that reputation even centuries after the South Arabian kingdoms had disappeared.
 

Bibliography

Beeston, A. (1937). Sabaean Inscriptions. Oxford: Oxford University Press. Nebes, N. (2011). “The Martyrs of Najran and the End of Himyar: on the Political History of South Arabia in the Early Sixth Century” in The Qur’ân in Context. Edited by Angelika Neuwirth et al. Leiden: Brill.
Robin, C. (1991). “Quelques épisodes marquants de l’histoire subarabique”. In Revue du monde musulman et de la Méditerranée, 61.
–––––– (1991). “Les langues de la péninsule arabique”. In Revue du monde musulman et de la Méditerrannée, 61. Smith, G.R. (1960) “al-Yaman” in Vol 9. of The Encyclopædia of Islam, New Edition. Leiden: Brill.
Stein, P. “The ‘Ḥimyaritic’ Language in preislamic Yemen – A Critical Re-evaluation” in Semitica et Classica, 1-1 (2008).
–––––– “Ancient South Arabian”. In The Semitic Languages – An International Handbook. Edited by Stefan Weninger. Berlin: Walter de Gruyter GmbH.
Vanhove, M. (1999) “Les dialectes arabes des régions sud, centre, et est du Yemen: perspectives de recherches". In Chroniques yémenites.