When did people learn that boiled water was safer to drink than right out of a river? They clearly knew that drinking beer and wine was safer since antiquity, and that if you drink upriver the water was less likely to make you sick. But was there a moment when they began sterilizing their drinking water? OR was it simply a byproduct of making teas and other infusions for culinary purposes?
that if you drink upriver the water was less likely to make you sick
True. Even better, go to the source. The less time the water spends flowing, the less opportunity there is for it to be fouled.
They clearly knew that drinking beer and wine was safer since antiquity
Not true. I'm afraid you're starting from a false premise here. I have a post that addresses this misconception, and while it's mainly addressed towards the Middle Ages, water remains water no matter what year it is in the world.
As we can see from the post, the Medievals already take it as granted that boiled water is indeed safer. This really is more of a question for r/AskAnthropology. I'm honestly not trying to be snarky here, but you're really asking "how did people figure out cooking". It's the same principle - just as food needs to be cooked before becoming edible, some water needs to be cooked before it's drinkable. (u/sunagainstgold has multiple times cited a letter that points this out.)