Martial, a Roman poet, wrote about a youth who was killed by an icicle that fell from a leaky aqueduct and pierced his jugular. See Martial 4:18. This would indicate that the aqueducts did, in fact, freeze over.

This was probably less of a problem than you might imagine. A working aqueduct would always have swiftly flowing water in it, and flowing water does not freeze as easily as still water. However, in very far northen parts of the empire, chiefly in what is today Germany, the Romans built the aqueducts partially below ground so that the water would be well above freezing temperature. The best-known example of this is the Eifel Aqueduct that supplied water to what is today the city of Koln.

Furthermore, very few of the Imperial-era cities large enough to need aqueducts were in places that experienced substantial periods below-freezing temperatures. Most of the largest cities in the Empire were in the Mediterranean Basin.