What would, say, a farmer in Central Europe in the ninth to fourteenth century drink on a cold evening or when he otherwise felt like something that was neither beer nor (expensive) mulled wine? Did people use local plants for tea regularly, or was there no social equivalent?
Herbal teas (or tisanes) are a very ancient practice all over the world, but predominately for medical or ritual use. Herbs, roots, and other plant products were generally assigned medicinal qualities (some legitimate, some spurious at best) and consumed as such. I don't know of any regular consumption of non-alcoholic boiled plant beverages for quenching thirst.
The most typical hot beverages that you would have on a cold evening would still be alcoholic! Consumption of alcohol like mulled wine, small beer, or cider in moderation was acceptable during the work day, and everyone would drink it in place of plain water if they could. The Romans in particular loved heating wine or mixing it with honey, herbs, and spices like the popular conditum paradoxum.