There are multiple factors at play in this.
Firstly, the Arabian Peninsula had not been unified, leading to the perpetual warring between various tribes. If you know anything about Mongol history, than you know what sort of warriors this sort of perpetual warfare can produce. Once united by Muhammad under the banner of Islam, these separate forces were able to focus their abilities outwards. Combined with the tradition of martyrdom and belief in the will of Allah within Islam, you have an effective fighting force. For example, at the Battle of Badr in 624 CE a group of 300 muslims defeated force nearly 3 times larger. This was seen as divine intervention and gave impetus to the Muslim forces. Fighting spirit should not be underestimated.
Secondly, the Byzantines and Sassanids had been weakened by decades of skirmishes and mismanagement. There were a series of wars between the Byzantines and Sassanids between 540 and 629, which resulted in broken armies and destroyed spirits. The situation was ripe for the up-and-coming Islamic Empire to strike at.
Also, the Byzantines and Sassanids relied on Arab client kingdoms, such as the Lakhmids and Ghassanids, to control the Arab populations along the border with the Arabian Peninsula and maintain a buffer zone around their empires proper. When these tribes finally fell, the Arabs of the region had little difficulty shifting their allegiance to the new Islamic empire, facilitating the conquest of large amounts of territory. They had little connection to the overarching Byzantine/Sassanid Empires and ethnically identified with the Arab Muslims.
The Islamic Empire was also, for the large part, benevolent rulers. They did not force conversions to Islam so long as you were Christian, Jewish, or (sometimes) Zoroastrian (although it was favored by institutional constraints like the necessity of paying the dhimma tax if you were a non-Muslim Person of the Book) and originally saw themselves as a separate military caste over the top of the native population. Again, this facilitated the ease of conquest and meant that they did not have to leave large armies behind to subdue restless populations.
If you have any specific questions I can follow up when I have some more time and better access to my books later today.
EDIT: I found a passage in Lapidus that perfectly sums this all up. Im attaching it for if you would care to read it.
The reasons for the relatively rapid success of the Arab-Muslim conquests are not hard to find. The Byzantine and Sasanian empires were both militarily exhausted by several decades of warfare prior to the Arab-Muslim invasions. The Christian populations - the Copts in Egypt, the Monophysites in Syria, and the Nestorians in Iraq - all had long histories of troubled relations with their Byzantine and Sasanian overlords. Their disaffection was important in the cases where Christian-Arab border tribes and military auxiliaries joined the conquerors and where fortified cities capitulated. The conquests, then, were due to victories over militarily weakened powers, and were consolidated because local populations were content to accept the new regime. The conquests were further secured by a large migration of Arabian people. With the defeat of the Byzantine and Sasanian empires, a frontier between populations broke down, leading to a massive movement of peoples from Arabia into the lands of the Middle East.
Most of those topics I had already hit on above, but I believe it provides a nice overview of the answer.
Sources
Hourani, Albert. A History of the Arab Peoples. 1991.
Lapidus, Ira. A History of Islamic Societies. 34. 2002.
For a more 'revisionist' take on why the Arab conquests succeeded (as compared to /u/frogbrooks' answer), take a look at my answer here. I took the position that there wasn't anything particularly special about these Arab armies, but this is just one viewpoint. Some western scholars are more sceptical of later Muslim sources and the stories of their ancestors' exploits they recorded, preferring instead to use primarily contemporary evidence from the seventh century. Others are more generous in their assessments, but there is no real way to say which 'side' is more correct than the other. For a question as complex as this, there is no 'right' answer, but instead different interpretations of the available evidence, so I wouldn't take any answer, including mine (which is the product of my own biases and interpretations), at face value or see them as representative of any sort of a consensus amongst academics! A recent book I can recommend on this topic however is Robert Hoyland's In God's Path: The Arab Conquests and the Creation of an Islamic Empire (2014), which provides an excellent narrative of this fascinating period, albeit also from a more 'revisionist' perspective - you can for instance compare his account with the more 'conservative' narrative found in Hugh Kennedy's The Great Arab Conquests: How the Spread of Islam Changed the World We Live In (2008).
Additionally, my comment on how much 'preparation' the Roman and Persian empires had before the conquests might also be useful, if only for my very brief discussion of how Arab raids were already happening in the 610s, well before the beginning of the Arab conquests. Finally, because I didn't discuss Persia in too much detail, this essay provides a very useful overview of the extent of our knowledge. The seventh century is a fascinating period, but for so many things we simply lack the sources to say anything for certain!