Hello! I am not an historian but I am a philosopher. My answer might be a bit different but I can give you insight regarding what philosophers of the time had to say on the topic. My response will cover 7th century BC to 4th century BC in Ancient Greece.
Presocratics were the philosophers that came before Socrates, Plato and Aristotle. The first notion "close" to infinity we can find are in a fragment from Anaximander. For most pre-socratics, it was not call infinite but ἄπειρον; apeiron. Or in English, unbound.
For Anaximander, apeiron was something invisible, timeless, and infinite that formed the basis of our universe. It has always been there, will always be, is unquantifiable and unattainable for mere mortals.
Apeiron is also considered "without beginning nor end" and to guide the universe and its "flow".
Now, most importantly, Anaximander does not represent all of Ancient Greece. Moreover, I know nothing about other ancient civilisations like let's say the Sumerians or Egyptians.
If I go back to Greece though, some other philosophers such as Aristotle believed the world to be finite. For him, the universe was finite and had 5 elements and 2 forces: water, earth, fire, air, aether and the two forces were levity and gravity.
The stoics believed the universe was finite but surrounded by infinite void.
And I could go on and on and cover other philosophers.
But it is difficult for me, I think you will realise, to really say what Ancient Greeks specifically believed. There were many competing ideas. Possibly a classicist or historian would know if there was a mainstream view that prevailed all throughout time in Ancient Greece but from the point of view of philosophers there was definitely many ideas of what the universe was like.
As for infinity, I hope my answer regarding Anaximander was helpful. I don't know what was up on the mathematical side of the concept but there definitely was something akin to infinity in the various cosmologies.
Lucretius ponders an infinite universe in the first book of On the Nature of Things. Also gravity and atoms - basically he was on the right scientific track. This would be written sometime before 55 B.C., and it introduced Epicurean philosophy to the Romans. Epicureanism dates back to around 307 B.C.