Does it make sense that when I think about antiquity I think about great infrastructure in every way and birth of democracy and so whereas Middle ages seem to be the worst time people ever had? Or is it just thinking about antiquity I imagine these good times 480-430 bc or so and the rest also was terrible? But if the Greek and roman empire have been such good places, would someone wonder about the main reasons how middle ages could follow?
MFW people go off on the 'bad Middle Ages' kick again
Basically, yes. You've fallen into the admittedly very common pop-history trap of "Classical Good, Middle Ages Bad", overlooking any of a dozen factors that life was not good for the average working stiff of a Classical Greek polis or Imperial Roman city, and similarly overlooking any of a dozen factors that life was not bad for the average working stiff of Medieval Siena or Exeter.
I commend to your attention my Middle Ages Are Best Ages post compilation, whose name should speak for itself. Start with that one-post overview in the first thread, then consult the other posts as your preferences take you.