Specifically related to Spain, The Rise and Fall of the Great Powers by Paul Kennedy has a good analysis of the Spanish economy and its many shortcomings.

Basically, I believe the short answer is possibly, but not because the colonies were economically unproductive. They produced huge amounts of ready gold and silver for the Spanish crown, which allowed it to function as a global power for a couple centuries.

A little background on Hapsburg Spain is necessary here. The Spanish empire was dependent on a small percentage of its population, the middle class of Castile, for much of its normal tax revenue, as traditional separation of power which prevented the Crown from leveraging taxes on Aragon and other provinces. Coupled with a preferential system for the nobility, and a regressive tax system which discouraged trade throughout the empire, and the Spanish empire extracted an inordinate amount of money from its fledgling productive middle class, which stunted growth over time.

This meant that tax revenue was mostly stagnant for the Spanish empire despite increasingly vast commitments throughout the world, be it funding wars against Protestant German princes, defending against Turkish incursions, invading the Netherlands, or maintaining its overseas holdings.

As such, there were times where the Spanish throne owed moneylenders 30 years of normal revenue, which could only be paid through extraordinary methods, such as selling off crown lands, seizing land from political opponents or the Church, and similar unsustainable practices. Thanks to Spain's vast commitments, these unsustainable practices occurred with regularity, leading to loss of normal revenue and increasing loans.

The Spanish crown was rather bad at finance, often delaying payments of interest and on occasion declaring bankruptcy rather than paying the massive loans it floated to pay for wars. This meant that Spain paid higher interest rates than most other nations, despite being far larger (militarily and economically) with greater income than say the Netherlands, who could get loans with relative ease due to a reputation for prompt repayment.

Against that background, the Spanish economy was ripe for inflation, due to government spending, and stagnation, as its productive classes were taxed to a punitive degree compared to its aristocracy, while inefficient practices, such as protectionist guilds and internal tariffs, were implemented for various reasons, ranging from a suspicious and distrusting government to short sighted tax policy.

As I understand it, the primary allegation for why the Spanish colonies were an economy liability was that they caused inflation throughout the empire. While this was partially true, as the money supply was growing, governmental policy was a greater issue here, as it poured more money into the Spanish economy than the treasure fleets, and at a greater rate. There was a time when decades of American silver and gold was promised to moneylenders, that money already spent on the army and navy.

You could argue that the colonies enabled the Spanish government to spend lavishly, which allowed them to push their economy to the brink of destruction, but it isn't that different from other sources of extraordinary income that Spanish crown possessed. Attribution here, in my opinion, is more towards overreach by the Spanish crown, and its massive amount of borrowing and spending, rather than the colonies for existing and producing gold.

I'm not familiar enough to feel comfortable talking about the colonies towards the end of the Spanish colonial empire, as it is possible that the cost to put down rebellion increased then, but for a couple centuries, the Spanish holdings in America allowed it to pay off loans that were necessary to function as a global power. Due to fiscal and monetary policy, these loans were damaging to the Spanish economy, but in and of themselves, I believe that the Spanish colonies benefited the Spanish economy, and, better managed, the income would have been a boon to Spanish growth.

Followup question if I may: Who ended up paying the cost of these colonial empires, and who would benefit most from it?

As a follow up question: Why did Germany take colonies when these colonies cost Germany more than it got from it?