That's quite a loaded question because it presumes that the conquest of the Inca and Aztec Empires was easy. It was not.

The first point that I need to establish is that there were far more than just a few hundred men on the Spanish side. Cortes repeatedly received reinforcements throughout his campaign. First, he was able to co-op the expedition of Panfilo de Narvaez, wining control of over 1000 men. According to Juan Cano, the Narvaez Expedition was composed of over 1600 men, meaning Cortes may have commanded about 2000 men, one of the largest forces the Spanish fielded under a single commander. Even after La Noche Triste Cortes continued to receive reinforcements, rebuilding his army from over 400 to over 900 men. All in all, as many as 3000 Spaniards may have fought during the Conquest of Mexico. It is similar for the Conquest of Peru. We are all familiar with the capture of Atahualpa by about 160 Spaniards. But we forget that Diego de Almagro arrived with reinforcements shortly afterwards. After the Inca's wealth was revealed Conquistadors from all over the Americas flooded into Peru in their thousands.

And that's just the Spanish troops. The Conquistadors were almost always accompanied by thousands of local allies. Cortes received assistance from Tlaxcala, a rival to the Mexica, who provided his troops with reinforcements, supplies, medicine, and military intelligence. In Peru, Pizzaro was greatly aided by the Incan Civil War, which prevented the Inca from forming a unified defensive strategy against the invaders. Political infighting continued amongst the elite even as Pizzaro entered Cuzco. The disruption threw the political system into chaos and created or exacerbated local feuds. The end result is that the Spanish were also to form some alliances, sometimes with local people subjugated by the Inca, or by different royal Inca factions, hoping to use the Spanish as mercenaries. These allies carried the bulk of the war effort.

Now for the role of disease. I think you underestimate how dangerous plagues were, because there are political consequences that follow the disease. Both the Inca and Aztec Empires were actually quite young. Thus, they had not laid down deep political roots yet. They both relied mostly on hegemonic systems of intermarriage, diplomacy, bribery, threats, and the like, which were actually quite sophisticated in detail. Nevertheless, they were far from complete when the Spanish arrived. Disease proved very dangerous to this type of political structure, as it broke these systems apart, causing infighting. Every part of these Empires went through a succession crisis within a single season, with the Inca Civil War being the most prominent example. However, the Mexica also had similar problems, as disease cost them important allies who either turned on them, or were too weak to contribute enough troops to aid them.

There were also a few 'tactical' decisions that aided the Conquistadors. Well, more like dirty fighting. In May of 1520, Pedro de Alvarado massacred a large group of Mexica nobles at a religious gathering, possibly killing 3000 men. After this, the Spanish took yet more nobles prisoner, executing man when the fighting went badly for them, possibly including Motecuhzoma. In Peru, Pizzaro was able to take Atahualpa prisoner relatively easily, because the Inca had arrived without weapons. No one knows exactly why they did this, but it is evident that they did not come expecting a fight. Basically, some of the Spaniards greatest victories were not fought against prepared enemies. They were little more than massacres. I should note that these two incidents were not unique. Conquistadors frequently befriended American leaders and then betrayed them, and they often attacked civilians to terrorise their enemies. Consequently, many Spanish victories were not really victories, but are often remembered as such.

Even then the battles for these two empires were extremely difficult, and often ended in defeat for the Spanish. On La Noche Triste the Mexica ambushed and killed between 600 and 1000 Spanish as they attempted to flee Tenochtitlan. The subsequent battle for the city dragged on for moths. It took 73 days of continuous fighting in the streets of the capital, and the Spanish had to level almost every building, before the Mexica surrendered. The Inca also proved more resilient that typically imagined. Manko Inca, who succeeded Atahualpa, lead a rebellion, defeating the Spanish in several battles before retreating to Vilcabamba where they held out until 1572. We can extend this to fighting in other parts of the Americas. The Calusa defeated Juan Ponce de Leon and the Pueblo nations drove the Spanish out of New Mexico in 1680. In Mexico, the fall of Tenochtitlan did not end the fighting, as the Spanish were soon embroiled in the Mixton War. There was also resistance from other parts of Central and Southern Mexico. In South America the Mapuches continually defeated Spanish attacks, and eventually outlasted them. They were not conquered until the 19th century. The Maya also resisted with incredible ferocity. The Montejo family needed five wars to conquer the Yucatan, and never really ruled the western portion of the peninsula. The Guatemalan war dragged on for over a decade, and most of the fighting was done by Mexica and Tlaxcalan allies anyway. The last Maya stronghold, Tayassal, did not fall until 1697.

Basically, the Aztec and Inca Empires did not go down easy.

References: Seven Myths of the Spanish Conquest by Matthew Restal Mexico and the Spanish Conquest by Ross Hassig Indian Conquistadors: Indigenous Allies in the Conquest of Mesoamerica edited by Laura E. Matthew and Michel R. Oudijk Fierce and Unnatural Cruelty by Inga Clendinnen