Yes, it happened, and fairly often. We tend to only remember the successful colonial attempts, with the possible exception of the lost colony of Roanoke entering the popular consciousness for its mysterious disappearance. In North America alone Spain, France, and England all lost several initial colonies, and failure was more common than success in those early years. Remember, colonists were invading someone's home. The lasting European settlements persisted due to the tolerance and support of indigenous peoples, food trade to survive the initial lean years, and expert geographic and political advice to navigate this New World.

Spain's initial colonial settlements in North America were met with a series of unfortunate events. Juan Ponce de Leon’s second journey to Florida ended in disaster shortly after landing on the Gulf Coast. Calusas attacked his party, wounding Leon with an arrow. The entrada limped back to Cuba, where Leon died of his wounds. Lucas de Ayllón mortgaged his fortune to mobilize a group of 600 colonists to head toward the U.S. southeast. He established San Miguel de Gualdape, the first Spanish settlement in what is now the United States. The colonists arrived too late in the season to plant, and fell ill, likely due to contaminated water sources. After Ayllón succumbed to illness, the colony fractured and abandoned San Miguel. Less than 150 colonists survived to limp back to Hispaniola.

After losing an eye fighting Cortés at Cempoala in Mexico, Narváez was appointed adelantado of Florida. His unfortunate decision to split his land and sea forces after landing near Tampa Bay was but one of many disastrous mistakes. Hunger, hostilities with the Apalachee, and illness diminished the strength of the land forces, who failed to reconnect and resupply with their sea-based comrades. Narváez decided to skirt the gulf coast back to Mexico, and died on a make-shift raft blown out into the Gulf of Mexico near Galveston, Texas. Only four men, including the famous Alvar Cabeza de Vaca, survived the final overland journey through Texas and into northern Mexico.

Hernando de Soto survived the conquest of Peru, only to die on the banks of the Mississippi after pillaging his way through the southeast. The exact location of his grave remains unknown and the tattered remnants of his forces limped south to the Gulf of Mexico. Even decades later, more disaster followed Spanish attempts to extend their frontier northwards. The Pueblo Revolt kicked Spain out of New Mexico, rolling the frontier southward for more than a decade. The attempt to establish a mission near the James River in Virginia ended abruptly when their kidnapped translator escaped and returned with a war party that killed the Jesuit fathers.

The French Huguenot colony of Charlesfort in northern Florida was abandoned in 1564, and the subsequent colony at Fort Caroline was attacked by the Spanish, with most of the inhabitants massacred after the fort was taken. France also lost several first colonial attempts in Canada including the short lived settlements at Sable Island and in Acadia.

The absolute disaster of the first English colony, a mining endeavor in the Arctic at Meta Incognita in 1576, is all but forgotten. Everything went wrong with this colony, from the terrible settlement placement, the fact they ended up mining tons of useless mica instead of gold, and the abandonment of a handful of settlers who were out on a hunting trip when leadership abruptly decided to go home in 1578. Seriously, the local Inuit couldn't believe the whole thing happened. In 1859, when Charles Francis Hall went looking for the lost Franklin Expedition, he interviewed Inuit historians who told the story of the how long ago they helped the abandoned white people for a time, before they eventually built a makeshift boat to sail back home and were never seen again.

If you want one source that talks about all these catastrophes, We Could Perceive No Sign of Them: Failed Colonies in North America, 1526–1689 is the book for you. The authors are historians, and I do fault them slightly for short-changing the anthropology and indigenous history portions of these colonial failures, but the book is a great place to dive in and learn more.