How was drinking water distributed during ancient Rome?

Pompeii provides the most complete evidence, but it is worth noting that enough fragments can be found from other cities to lead to the conclusion that this was fairly common across the empire, so we don;t have to worry about falling into the Vesuvian trap. Another note is that it is common to read that fresh water from the aqueducts was something only the elite enjoyed--this is a simplification of a complex archaeological argument. As a matter of fact, the amount of water per person per day pumped into the city of ancient Rome was significantly more than the modern city of Rome--200 gallons. Much of this would indeed be earmarked for agricultural use, fountains and the baths, but it is virtually impossible to cut this number down enough that there isn't more than enough for everyone. Aqueducts are also found throughout the empire, not just in Italy and the Roman colonies.

Water was piped into the city by aqueducts. It is important to remember that most of the stretch of the aqueduct would not look like the famous Pont du Gard. Most of it would be at ground level or slightly underground. Most of them were also only a few kilometers long, the distance from the nearest water source (usually a nearby hill) to the city. When it actually reached the city it would be branched off into water castles, which were strategically distributed throughout the city. In some areas, most notably North Africa, they might also go to massive underground cisterns. From the water castles they would be sent to fountains--the tap is modern, in ancient times the water was continuous. These were common throughout a city and could be easily accessed. They may have also played a role in street cleaning--by continuously overflowing they washed water down the street, potentially sending refuse into the sewer systems.

Mary Beard has a good account of this in Fires of Vesuvius, about Pompeii.