The short answer is that the Byzantines and the Persians were exhausted after twenty six years of war (which only ended four years before Muhammad's death), whilst the Arabs were led by able leaders and had better morale. This is also a very big topic, so although I'll try to cover all the important aspects, there's always room for others to improve on this answer!

The Byzantine-Sassanian War of 602-628 was a key event in the history of the Near East, and if it wasn't followed by the Arab Conquests, it would have been the defining event of Late Antiquity. During the war, the Byzantine administration slashed the wages of soldiers and bureaucrats, melted down 'volunteered' church silver to pay for the war, and the citizens of Constantinople made do without the traditional free bread dole. From the mid-610s onwards, the empire lost its wealthiest provinces in the Levant and Egypt to the Persians, whilst Anatolia was regularly raided. The situation in the Balkans wasn't great either, since the Avars and Slavs, both occasional allies with the Persians, pillaged as far as Greece, with some tribes even settling in the area. Western Byzantine provinces such as Italy and North Africa were almost non-entities at this point, since the administration could do little to support them and vice versa. Constantinople itself was besieged in 626 and Emperor Heraclius could do nothing to help his capital - he was leading the one functioning army he still had in a risky campaign in the Caucasus Mountains. The fact that he was personally leading an army was exceptional enough, as no emperor had done that for centuries. Through luck and military skill, Constantinople repelled the Avar besiegers and Heraclius won several miraculous victories when he led his army deep into the Persian heartlands, causing an internal Persian coup that ended the war.

So though the Byzantines 'won' the war, it was a close shave and it easily could have gone the other way. The situation afterwards wasn't much better. The Persian armies in the Near East were only evacuated when Heraclius negotiated with their general, as he couldn't get them out otherwise! Even then, Byzantine armies were returning to regions which hadn't seen Byzantine rule for nearly a generation (610s to 628). The Arab confederations that guarded the desert frontiers were now of doubtful loyalty too, as they may well have been employed by the Persians during the occupation, whilst large chunks of the population, Miaphysite Christians and Jews in particular, were generally treated quite well by the Persians and so cannot have looked forward to being once again ruled by an empire that saw them as heretics.

Not surprisingly, the Persians fared even worse. There was now a four-way civil war between rival claimants after various able candidates for the throne conveniently died. Its armies were uncoordinated and more focused on their internal enemies rather than defending their frontiers. The Persians used to have a confederation of Arabs that defended their border with Arabia too, but the Lakhmids' power was destroyed by Khusro II during his last great destructive war, so the border was now more permeable than ever. According to recent studies, Persia was also relatively decentralised and had very sharp divides between different classes, so the fact that they lasted as long as they did (Yazdegerd III was killed only in 651) is quite astonishing.

When Muhammad returned to Mecca in triumph in 630 (or maybe not if you follow Howard-Johnston's argument), Heraclius was at height of his power, as he had just returned the True Cross to Jerusalem in a grand procession at the same time, hoping that he was now able to slowly rebuild his empire. Unfortunately, Heraclius also issued a number of damaging edicts. Jews were to be forcibly converted across the empire, and Heraclius began to push for the unification of the various Christian Churches in the Near East. To bridge the gap between Chalcedonian and Miaphysite beliefs, he and the like-minded Patriarch Sergius of Constantinople created Monoenergism and later Monotheletism, compromise Christological formulations that ignored the issue of how many natures Jesus had altogether, focusing instead on how Jesus possessed 'one energy' and 'one will' respectively. This worked to an extent, but it also made some Chalcedonian and Miaphysite Christians rather angry, since this policy was imposed with force if people did not accept it willingly. From the sources, we hear of brutal persecutions in Egypt and religiously-inspired revolts in North Africa and Italy. Heraclius' policy, though perhaps well-meaning, did a great deal of damage to the empire. I talked a bit about the impact of this on the conquest of Egypt here, so I won't repeat it, but I think it is quite obvious that given the choice, oppressed Miaphysites would not necessarily want to fight to the death against Arab soldiers, when Islam promised to respect Christianity of all stripes equally. Islam also had some connection with Judaism, though the extent is debated by modern historians, (see Hagarism by Crone and Cook for its most forceful iteration, Donner's recent Muhammad and the Believers provides a much more convincing argument!) so I think it is quite likely that many Jews threw their lot in with the Arabs too, rather than support their persecutors.

The opposition wasn't just weak because of religion of course. I think simple exhaustion played a big role. Though initially armies gave battles, even winning victories on occasion (the Battle of the Bridge in Iraq being the best example), and fortresses were often able to resist for long periods of time (particularly Caesarea in Palestine and Babylon in Egypt), after a few defeats, many important cities, such as Jerusalem and Alexandria, surrendered without a fight. This wasn't that exceptional, since the same thing happened when the Persians swept away the Byzantines - most people saw very little point in fighting to the bitter end if they knew it wasn't going to make a difference. The same was true in Persia, where after the disastrous defeat at al-Qadisiyyah, the people of Iraq seemingly just gave up on fighting. After all, there was no great danger to their properties provided that they pay the demanded tax to their new rulers. The fact that Arabs formed significant minorities in both Syria and Iraq must have helped too, as Islam gave a sense of unity to Arabs everywhere, a group long dismissed by imperial powers as mere barbarians, whereas both Persians and Byzantines had their own internal divisions.

Lastly, I don't think we can characterise Arabia as a backwater region. Parts of Yemen for instance were reasonably fertile, whilst even Bedouin nomads would have come into contact with the great empires around them. Christianity, Judaism and Zoroastrianism all had their adherents in the peninsula, which combined with trade networks and military connections (such as the confederations that guarded the imperial borders mentioned above), meant that Arabs were fully plugged into the Late Antique world. They would have been aware of how weak Byzantium and Persia were, how religious unrest was rife in Byzantine territories and of the Persian civil war, all of which made the two empires tempting targets for Arabs used to tribal warfare and political machinations.

Their success was not inevitable by any means, but I think in the circumstances, the Arab Conquests were not exceptional. Whilst the Byzantines and the Persians had more people, more wealth and more centralised forms of government, during the Arab Conquests they were not able to use their resources effectively. Morale was low and the leadership was exhausted, if it still existed. The Arab armies however were the opposite.

I missed out a lot of the nuances of the Conquests, especially the historiography and the Arab perspective, so feel free to ask me to expand on where I wasn't clear :)

Sources:

R Hoyland, In God's Path - a very readable and up-to-date narrative of the Arab Conquests by an excellent historian of early Islam, can't recommend this enough.

J Howard-Johnston, Witnesses to a World Crisis - a book covering the Byzantine-Sassanian War and the Arab Conquests, but source analysis is where it shines. It argues for a slightly altered chronology for the Arab Conquests as well, so this is a very interesting read.

R Hoyland, Seeing Islam as Others Saw It - a collection of all contemporary sources on early Islam, quite possibly the most useful book for studying this fascinating period.

While I cannot speak for manpower, the Islamic forces managed to conquer Persia and a good chunk of Byzantine holdings by way of superb discipline through religious zeal, excellent leadership in desert warfare, and the fact that Persia and Byzantium were both utterly wasted and gassed by years of warfare. Heraclius' war against Sassanid Persia had gone on for more than a quarter century, and while he was victorious, he not only left Persia ripe for the taking from outside invaders, but also left his own armies too exhausted and weakened to put up a fight against the Muslims and their extremely talented military leaders. Had Heraclius died upon the conclusion of the last Byzantine-Sassanid war, he might've been considered one of the greatest of the Eastern Roman emperors. Unfortunately, he lived just a bit too long, and would only see the beginnings of what would end up amounting to massive losses against Islam over the next few decades.

I'd imagine someone with more detailed sources than mine can expound upon this.

Source: J.J. Norwich, "History of Byzantium"

Edit: Read the answer by /u/shlin28 since it's way, way, way better than mine.

You might be interested to read Tom Holland's recent book "The shadow of the sword". It's intended for a popular audience and, as a complete non-specialist, I found it enthralling.

To some extent he extrapolates from late Roman history into the rise and early years of Islam (for which apparently very few pertinent documents survive). One of the aspects I found most fascinating, and that relates directly to your question, was that various of the Arab tribes had been working as mercenaries for either the Romans or Persians for many years and so would have been well placed to take advantage when the two empires finally exhausted one another.

I know this book got some negative criticism from certain historians - I'd be very interested if any historians in this thread were to comment on that.

Not sure if this is too remote, but kind of a follow-up question:

The arabs were in a position of power in the Levant after their conquest. How did the emerging Ottoman Empire take over the leadership position in the muslim world? Is it to do with the Mongol expansion?

What was the relationship between arabs and other muslims like?