First thing's first, the first major issue with South America breaking up has to do with differences in the manner of gaining independence. Brazil gained its independence much more peacefully than the Spanish-speaking nations. Much in the same way that leaders in Spanish America took advantage of Spain's weakness as a result of Napoleon's Iberian invasions, the independence of Brazil was the result of that same historical episode. The first emperor, Dom Pedro I, was the son of the king of Portugal who was left to rule Brazil as regent, but then decided to support Brazilian independence. This is not to say that the struggle for Brazilian independence was totally peaceful, as there was some troop activity but the Spanish-American wars were far more extensive. Therefore in the independence movements, a division between Spanish- and Portuguese-speaking territories was somewhat inevitable.

However, as you mentioned, geography was a major dividing factor. As J. Christopher Herold put it,

One of the main causes of division was simply the immensity of the territory and the formidable geographic obstacles separating its component parts. Communications between Madrid and Buenos Aires, between Madrid and Mexico were easier than between Mexico and Buenos Aires.

The Spanish territories in the Americas were so extensive that there were massive cultural differences between colonials. Furthermore, the several main independence movements in Spanish America, based in each of the several Spanish viceroyalties, were all responses to relatively local issues pertaining to popular sovereignty as well as socio-racial equity. These factors made a unified independence movement, like the one seen in British America with the creation of "continental" bodies and forces, nearly impossible. Therefore, the separate independence movements resulted in the creation of their own separate countries in the aftermath.

There were also a number of racial differences among the different regions of Spanish America that became another factor that further divided the total population. While there was a great amount of racial mixing among European and indigenous populations in Mexico, Central America, and the Andean region, such was not as common a practice in the Southern Cone of South America. This can be seen even today in the varying forms of national selfhood, expressed through mestizaje (the mixing of European and indigenous) in places like Mexico, but expressed through celebration of lo criollo (purely Spanish American) in places like Argentina and Chile. The varying levels in racial mixing in different regions resulted in cultural difference, making political consolidation and unification increasingly difficult.

There were in fact attempts at fostering a sense of Pan-Americanism by figures like Simón Bolívar, though. Inter-American conferences were held throughout the nineteenth century starting in Panama in 1826. But by then, the results of cultural difference and nation-building across the continent had done their job, making these conferences more about common defense and political cooperation. For further reading, you can look at John Charles Chasteen's book Americanos, as well as Marie Arana's book Bolívar. For the impact of the Napoleonic Wars on Latin America, J. Christopher Herold's book Age of Napoleon dedicates ample space for it.

When I was reading a book about Central America, (if I remember correctly 'Failure of Union' which is in this subs reading list) I read a good point: you can ask 'Why didn't Central America form a single, unified country', and that's a fair question, but you can equally ask 'Why would they?'. Apply this to Spanish America as a whole and the point becomes even more relevant.

Spanish America was, for a start, not just a single political unit. It was split up into a number of political, judicial, and administrative units. These divisions aren't exactly the borders you see today, but many current borders in the region (sort of) follow them. So why would someone in Montevideo think they have to be in the same state as someone in Maracaibo? They've never shared the same immediate government anyway.

It's not necessarily the case that people identified as a resident of this or that viceroyalty anyway. To further the example above, why would someone in Montevideo think they should be under the same government as Buenos Aires? Many of the new states had enough trouble enforcing a national government, making these states larger would no doubt just add another layer of difficulty.

As mentioned by the other answer, we are talking about a vast region too, spread across two continents. That can complicate things in any situation, but in a time without motorised transportation or electronic communication such difficulties could only really be managed with some degree of local autonomy. The Spanish empire managed this for a while, but they also had the legitimacy to do so. Similarly, the infrastructure that was there was not focused on linking up the different parts of Spanish America so much as linking those parts to Spain (to an extent).

So when the time came for independence (to simplify things), you didn't have one cohesive political unit seeking independence as a whole. You had a variety of different movements, at different times, who often fought for ostensibly different reasons. They fought mostly against loyalists from on the continent, not from Spain, and had to contend with a variety of local powers, conditions, desires, etc. So there was little reason to expect that they ever would be unified. To exaggerate a bit (or a lot), it's like asking why Canada and Denmark aren't the same country. It can be a little hard to answer beyond 'Why would they be?'

But of course there are reasons why you might think they would be. Beyond just that they're places colonised by the same power, mostly (but not entirely) speaking the same language. There was, and in a different way still is, some sort of common identity. Peru is very different from Mexico, but there are people in both who consider themselves Americans. There were people in the former colonies at the time asking the same question as you. Additionally, local/municipal identity existed, but at the time national identity was still being formed, and so was still fairly malleable.

And, contrary to what I wrote earlier, some of the new states did, at times, encompass larger areas than they did now. Montevideo was in fact under the same government as Buenos Aires for a time. Bolivia (once called upper Peru) was joined to Peru. Colombia, Venezuela, and Ecuador were all joined. Central America was nominally unified.

Of course, all these examples were failures. Uruguay has its independence, as does Bolivia. The former New Granada (Colombia, Venezuela, Ecuador) broke apart, and Central America was never really unified at any point. In all these cases, local forces won out against attempted unification.

I guess my point is that while there were views towards unification at various levels, more local interests were stronger in most cases. While it's hard to say much that applies to the whole region, the states that did form were probably about as large as could be expected. Going into it expecting that they should naturally have coalesced might give you the wrong frame of mind.