Since I'm interested in seeing this question get answered as well, might I propose breaking this up into a few sub questions so the answer can be found?
-What is currently the earliest evidence for humans distilling something so it contained alcohol?
-What is currently the earliest evidence for humans making drinkable remedies that involved mixing things into water or other liquids?
-What is currently the earliest evidence for humans squeezing fruits and/or other plants for their juices to drink?
While it will probably be harder to answer the second or third questions (I don't know how one would document the first evidence for such things - an archaeologist might know this), the first question I posed is probably a previously asked question somewhere around this subreddit. I think these kind of questions are your best bets for getting an answer to your original question though.
I think it first makes sense to think about what types of things do people drink that aren't water or water infusions (like tea):
Those are the main categories I can think of off the top of my head.
I also think we should define "start drinking" as widespread adoption in a culture.
With those parameters defined I think we can do a bit of historical deduction.
milk
Large milk-able mammals were domesticated starting in the Neolithic era so milk presumably could have the honor of being the first widespread drink besides water. Large mammals were first domesticated in south asia but were first raised as sources of meat. Archaeologists tell us that exploitations of dairy from mammals didn't start occurring until 7000 BC which is as early as some of the oldest know beverages, but I don't know if milk could be considered a widespread beverage. Especially in hot south asia, milk is a very unstable product from a microbrial standpoint and from a calorie preservation point of view it is in the farmer's interest to culture it with bennifical organisms as fast as possible to create relatively stable yogurts and cheeses. Milk probably would have been a nice treat but cultured (and thus congealed) milk is a more rational way to consume most of this product.
Fruit Juice
I think we can also rule out fruit juices. Primarily because most fruits are actually really hard to juice. Most fruits have a high solid to liquid ratio and require specialized pressing machinery to extract liquid from and presses that could juice something like an apple were a relatively late invention.
Citrus fruits are fairly "juicy" by fruit standards but they are also a relatively late in being domesticated. Wild citrus were mostly sour and to the best of my recollection the earliest domesticated sweet citrus would have been mandarins around 500 BC. That is significantly later than other beverages that were produced in china (which we will see in a moment).
The other "Juicy" fruit is the "wine grape". This fruit is both the sweetest and juiciest fruit that humans grow. It is also one of the first plants domesticated by humans so perhaps grape juice is was the first widely adopted beverage besides water? I think its safe to say no. Mostly because fresh grape juice would have only been available during a few week window each fall. And as soon as you pressed the juice it would almost immediately turn into wine or some other microbialy transformed state.
Wine/Beer/Mead
So that leaves the alcoholic beverages. Grapes, grains and honey were all used very early in human history and all have left an archeological record. Thus beer, wine or mead could potentially all be candidates for the first widely consumed beverage besides water.
As it turns out its kind of a tie. In general, early alcoholic beverages seemed to contain all three ingredients based on modern analysis of residue on pottery remains. And it kind of makes sense. Grain is abundant and can produce fermentable sugars via malting or saccrification via mold. Honey tastes good and is another sugar source (despite being fairly expensive). And the grapes are a briefly available sugar source, but more importantly they are a vector for the fermentation yeasts that convert sugar into alcohol.
It is an often repeated misconception that our environments are just swarming with airborne yeasts just waiting to land on a sugar solution to start fermentation. The realty is actually that the primary fermentation yeasts are not airborne and need some sort of vector to get into the solution. As it turns out the most common manner was that social wasps would land on damaged fruit to feast and in the process they would deposit yeast. This yeast can then started multiplying and produce alcohol as a form of chemical warfare. Because of this phenomenon it would very much make sense to throw some grapes into a grain sugar and honey solution to kick off the fermentation.
And in fact the earliest known alcoholic beverage that has left an archeological record is a grain/honey/grape mixture from 7000 BC in Jiahu China.
I think this could be a question for r/askanthropology. There is a similar question on there right now as well.
Besides alcoholic beverages and other non-alcoholic solutions in water, for which the first evidence we have is pottery (intended to contain liquids), there is also another naturally occuring liquid that can be drunk: blood. Drinking blood from living cattle is somewhat common among African pasturalists, and it is known from the Eurasian steppes as well. The earliest possible occurrence of blood-drinking could have been when humans switched over from scavenging to hunting (though small-scale hunting also occurs in our primate cousins), around one and a half million years ago or so. Besides this, wound-licking is also common among both humans and other animals, though its kind of a stretch to call this 'drinking'.