Why did people think that by the year 2000 we would have flying cars and jetpacks, etc?

Lack of understanding of technology, economics and politics. I would pause right there and ask who made the predictions? Was it experts in the field? Was it the lay press? Was it writers hoping to sell a story to Popular Science? Your expectations would vary depending on the source of the prediction.

Consider the space stuff you mentioned: space stations, moon bases, interplanetary travel. The rocket equation has been known for 200 years. There is a hard limit to the payload you can expect to carry with a certain fuel, which depends on the velocity of the exhaust gases, which depends on the energy contained in the fuel. Chemical fuels have limited energy, making space travel very expensive. The chemistry to understand these things was known a hundred years ago, certainly in the 50's - 70's when people were making wild predictions about moon bases and vacationing on the moon.

In the early days, it cost $20,000 to send a pound of payload into low earth orbit (LEO). Now the cost is down to about $3,000. Some experts say it can probably be brought down to $1,000 or perhaps a bit lower. But even at $1000/lb, it would cost $200,000 to send a man to LEO, more if you consider food and oxygen and life support. These are rare-government-project or super-rich-jaunt prices, not vacationing on the moon for the common man prices. It will never fall to such prices given the constraints of the rocket equation and chemical fuels. Scientists knew this when those speculative articles were written, or at least, the experts did.

There is some element of hopefulness there. Perhaps we'll invent nuclear rocket engines, perhaps some completely unknown technology will replace rockets. Usually this comes from the lay press, who don't understand practical problems such as how much a reactor weighs, how will it be cooled in space, what will it use for reaction mass, how to safely launch it so it doesn't crash and produce a radioactive cloud on earth, etc. And if you could solve all these problems, how much would it even bring down prices, would it be enough for the scifi future they predict?

Then there is the economics and politics. The Apollo program cost about $110 billion in current dollars, to send a few men to the moon for a few hours. How many tons of materials would it take to create and supply a moon base? How many dollars? How will the government and taxpayers justify trillions spent on such a moonbase, instead of taking care of problems right here on earth? What's the payoff, how far in the future is it, how do you sell a payoff that's 50 or 100 years down the line? For many speculative stories, the answer was simple, don't think about this stuff.