I know this question covers a long period of history. They often are covered in world history in the US during the 1400s but largely disappear from the history books into the 1500s and 1600s.
What events led to their general disappearance as other European powers rose and spread across the Western Hemisphere?
Finally a question I can answer! The reasons for the decline of Portugal from its height as the first colonial power to its situation today as, at best, a middling european power are varied and extend over centuries. First, Portugal had an inherent disadvantage to other colonial powers like England, France, or neighboring Spain; it's simply a smaller country. It had a smaller population and economy, which meant it had fewer people to send overseas, fewer funds to spend on them, and a smaller army to defend them with. The second blow to Portugal’s power was its union with Spain beginning in 1580 after the death of King Sebastian, and lasting to 1640 with Portuguese Restoration War. During this time under the Spanish crown, Spain's enemies became Portugal’s enemies, those being most importantly the Dutch and British, who now having the opportunity to do so, took many of Portugal’s colonies and trading posts in the Indian Ocean, such as Hormuz and Malacca, as well as making attacks into Brazil. This along with having to devote resources to Spain’s numerous wars during this period (including contributing ships to the infamous Spanish Armada) drained Portugal greatly, and from this point forward only declined relative to the rapid growth of the other colonial powers. In 1755 Portugal experienced one of the worst earthquakes in European history off its coast, resulting in the widespread damage and the near complete destruction of its capital and largest city, Lisbon. The Peninsula war in 1808 and the loss of Brazil (its richest colony) in 1822 Portugal further hurt Portugal, though at this point they simply cemented Portugal’s poor position. Lack of large deposits of coal or other resources meant industrialization came slowly to Portugal, which as the 19th century progressed only hurt it. During the scramble for Africa Portugal, despite having colonies in Africa for over 400 years by then generally just expanded around their existing holdings. Portugal in the 20th century was marked by the Estado Novo, a dictatorship lasting from 1933-1974, a dictatorship which saw general isolation from the world, except for fruitless, costly wars trying to retain their colonies.
In conclusion, (tldr) Portugal from the start was doomed to a certain degree to not achieve the same height of successes of its competitors, and despite its early start it was further set back and mired by numerous disasters and generally unfortunate events.
I hope this was sufficiently in depth, and while I'm sure I missed certain elements, I hope this gives you a good general insight on this topic.
Not to discourage other answers, but while you're waiting, you might want to check out The Decline of the Spanish Empire section of the wiki, which touches on both Spain and Portugal. You might find this post with an answer by /u/Chilangosta to be pretty helpful, although it's pretty heavily skewed towards Spain.
With so many deleted comments I'm not sure if this answer is up to standards, but I'll try anyway:
One could argue that such a small nation could never hope to maintain its lead at the forefront of colonial empires for long. Despite being the first, Portugal suffered from having a population too small to keep up with other powers such as France, England or even Spain, much like the Dutch Empire. With a small population, one simply couldn't project the power necessary to challenge competing colonial nations for too long. The Portuguese had adopted gunpowder weapons early on and were better equipped than their contemporaries but only for the very first years of their empire; this advantage would wear on as the technology spread and Portugal was surpassed. With neither men nor superior weaponry, Portugal's sole advantage was being the first.
From 1581 onwards, Portugal was in a personal union with the Habsburg king of Spain. Although this didn't mean that Portugal was annexed by Spain, its wealth would be used to fund Spanish wars and the country would be dragged into Spanish rivalries with other european powers, which disrupted the Portuguese trade (Up to this point Portugal had been a staunch English ally and it would resume to be so after the union with Spain ended). Their colonies, notably Brazil, where occupied by the French and Dutch during this period, taking the lucrative Sugarcane plantations there.
The Portuguese waged a lenghty Restoration War from 1640 to 1668, and a massive earthquake hit Lisbon in 1755, with deaths toll estimated as high as 100,000. Portugal already was a small country, and now it had even less men and had to completely rebuild their capital from the ground up. Later on Napoleon would invade the country and the royal family would flee to Brazil, indirectly bringing about the independence of that country, Portugal's major colony.
On the early days of their colonial empire and their sea route to India in 1498 the Portuguese enjoyed little to no competition, but by the 17th century, as you can see, it had to compete with the trade or directy emnity from the Dutch, English and French colonial empires. Its little country and army could not help but be eclipsed, as larger nations that could exert their power on a larger scale entered the age of colonialism.
Portugal was much too small. They were the first European country to start a naval trading empire, mainly through the sponsorship of Prince Henry the Navigator and their invention of the caravel (small, highly maneuverable sailing ship), but they could never really take full advantage of their position due to limited manpower and resources. Although in the beginning they enjoyed dominance on spice trade with India, the Spice Islands (now known as Moluccas), and China (as Muslim traders had to travel a treacherous, lengthly, and difficult journey by foot in comparison), they weren't powerful enough to compete with other European forces that soon joined the fray, such as the Spanish, Dutch, and English. They didn't have the resources or power to colonize the regions they traded with and further expand their trade empire, while later countries did.
From p. 408 in Western Civilization by Jackson J. Spielvogel:
Within a few years, the Portuguese had managed to seize control of the spice trade from Muslim traders and had garnered substantial profits for the Portuguese monarchy. Nevertheless, the Portuguese Empire remained limited, consisting of only trading posts on the coasts of India and China. The Portuguese lacked the power, the population, an the desire to colonize the Asian regions.
[...] Portugal's efforts to dominate the trade of Southeast Asia were never totally successful. The Portuguese lacked both the numbers and the wealth to overcome local resistance and colonize the Asian regions. Portugal's empire was simply too large and Portugal too small to contain it.
The Spanish soon followed through their exploration of the New World, and their greater resources enabled them to establish a much bigger overseas empire than that of the Portugese.
Portugal's main rivals, however, were the Dutch and the English, who were much better financed than the Portuguese, and they fought for control over the spice trade. Portugal, being a much weaker country, soon began to lose its trade empire. The shift in power first began when the Dutch seized a Portuguese fort in the Moluccas in the 17th century. From there, they gradually pushed the Portuguese out of the spice trade, and in the next fifty years, occupied most of the Portuguese coastal forts along the trade routes throughout the Indian Ocean (such as Ceylon and Malacca).
Furthermore, the Dutch were able to bring inland regions under their control in order to protect their position and maximize profits, unlike the Portuguese who mainly stuck to coastal ports and made agreements. The Dutch East India Company established pepper plantations on the neighboring island of Sumatra, bringing massive profits and increasing their wealth and influence considerably. By the end of the eighteenth century, the Dutch had brought almost the entire Indonesian archipelago under their control.
Furthermore, in Southeast Asia, where Portugal had previously established uncontested trade relations with mainland states such as Thailand, Burma, and Vietnam, other European nations soon began to actively compete for trade and missionary privileges as well, limiting Portugal's wealth and influence.
Sources: Western Civilization by Jackson J. Spielvogel, Brittanica Encylopedia
These answers are extremely shallow and mostly incorrect. Having said that I won't write a concise history of portugal here, instead I will link you to a book that explains this subject matter well: https://www.wook.pt/livro/historia-economica-de-portugal-pedro-lains/11532757
Sadly this book has only been written in Portuguese and I doubt there have been any translations of decent books on the economic history of Portugal.
To summarise however, the influence of the Jesuits (for example destroying the education system or causing Uruguay to side with Spain), the French invasion, the lack of a strong enlightenment and the power of the old aristocracy and clergy meant that Portugal had a terrible 19th century overall, despite being wealthy by the end of the 18th century. Portugal went bankrupt several times and the successive governments were unable to solve this issue due to gross incompetence.
Once the republic was established, Portugal had about 3 decades of anarchy and further destitution, only for the fascist regime to take over for another 4 decades. At this point Portugal was so poor that over 25% of the population could not read or write their own name and its immigrants were creating bidonvilles in France. In essence 2 centuries of terrible economic activity will do that to you.
Portugal's size is completely irrelevant in all of this as this is neither a necessary nor sufficient factor for economic prosperity or global influence, and Portugal was actually its wealthiest after the Restoration, not before. I suggest you read a good book on this matter if you're interested because you'll get an inaccurate summary at best from a reddit comment.
Hello everyone,
In this thread, there have been a large number of incorrect, speculative, or otherwise disallowed comments. As such, they were removed by the mod-team. Please, before you attempt to answer the question, keep in mind our rules concerning in-depth and comprehensive responses. Answers that do not meet the standards we ask for will be removed, and posters who break the rules of the subreddit admonished as applicable.
While we realize this thread is trending high in the subreddit, please keep in mind that those upvotes represent interest in the question itself, and it can often take time for a good answer to be written. We know that it can be frustrating to come in here from your front page and see only [removed] and this post, but we ask for your patience and understanding. If you are looking for some interesting content in the mean time, we hope you will check out our Twitter, the Sunday Digest, or the Monthly "Best Of" feature. It is very rare that a decent answer doesn't result in due time, so please do come check into the thread in a few hours. A Private Message to the Remind Me bot is a good way to remember.
Additionally, it is unfair to the OP to further derail this thread with off topic conversation, so if anyone has further questions or concerns, I would ask that they be directed to modmail, or a META thread. Thank you!