The first evidence of domestication of dairying animals arises 10,500 years before present (BP) in Anatolia with the domestication of goat, sheep, and cattle. These species spread to Europe 9,000 BP and to Northern Africa by 7,000 BP. Cows aren't the only species likely domesticated for their milk. In the Indus Valley dairying buffalo were domesticated around 4,500 BP, roughly the same time the yak was domesticated in Tibet. By the time we add donkeys (6,000 BP in Arabia or East Africa), camels (Central Asia 5,000 BP), and dromedary (Arabia 3,000 BP), you can see our species was really trying to unlock the benefits of domestication.

So, we like to domesticate large mammals. That doesn't mean we were drinking milk, right? Archaeological evidence of milk processing is present through residue analysis of ceramics from Anatolia 9,000 BP. In Europe 8,000 BP we find evidence of cheese processing, and analysis of dental calculus (that hard stuff that builds up on your teeth) revealed people were either drinking milk or some other lactase-rich drink, 5,000 BP. Ceramics in Libya point to milk processing 7,000 BP, and in Kazakhstan by 5,500 BP.

For added context, about ten years ago I would have tried to answer this question based solely on genetic information. Population geneticists have been deeply interested in the origin of lactase persistence (the ability to make the enzyme needed to break down lactose in milk into adulthood) in our species. At least five different times across the Old World, a mutation arose that allowed for lactase persistence. Those mutations were assumed to provide significant reproductive advantages to humans who could drink milk, given their high selection coefficient (measure of how intense natural selection is selecting for that gene). In fact, lactase persistence was one of the classic examples of positive selection in our genome. Unfortunately, that really cool story seems to be an oversimplification. Human populations lacking lactase persistence genes (like in Central Asia) still herd animals and drink their milk, and ancient DNA analysis of burials in places with currently high rates of lactase persistence (like Northern Europe) show the gene wasn't common before the Middle Ages. See this overview for a really cool synthesis of the archaeological and genetic data.