Note that this answer is for Medieval Sweden, and cannot be used for other locations.

As Olaus Magnus in his Historia de Gentibus Septentrionalibus (History of the Northern People, a collection of legends, cultural observations and history first printed in Rome 1555) notes, Swedish peasants had since heathen times a long tradition of cleanliness and bathing. Perhaps not in the sense that we consider it bathing, but still. Thwe word for Saturday in Swedish, Lördag, comes from Lögardag, or bath- (or wash) day. Tradition has it that one would wash one's entire body once a week, on Saturday.

Most likely, most (but not all) farmsteads had a bathhouse in Medieval Sweden. This would be a separate building kept apart from the other buildings of the farmstead due to the risk of fire. Bathhouses were used like modern saunas, where rocks were heated over fire and water then thrown on the rocks, creating a hot mist in which people sat on wooden benches and struck each other with bundles of birch twigs with the leaves still on, dipped in water beforehand. After sweating, you would pour cold water over yourself to rinse off, and then be clean.

Since many variants nits and lice do not like hot temperatures, it is quite possible that this method of bathing was more effective against them and the diseases they spread than what we think of as regular bathing. Combing out lice and louse eggs out of your hair or beard with a very tight-toothed comb after bathing would still be common.

Bathhouses took a special position in the Swedish Medieval County Laws. While not as sacred as the Church, it was still considered extremely underhanded to attack or kill someone on the bathhouse, where he or she would be naked, unarmed and defenceless. The older Västgöta law, the Bjärkö law and the Visby town law had double punishment for a crime conducted against someone in a bathhouse. The younger county laws even puts attacks in Church and in the bathhouse on the same level (some of them also includes attacks on someone sitting in the privy - the Medieval Swedes liked their privacy, it seems).

A bit of a side-track here - I know you asked about rural bathing, but bathhouses in towns were protifable establishments, and the conflict on wether burghers and towndwellers had the right to construct their own bathhouses or were required to use the public ones were common in Medieval Sweden - bathhouses were commonly willed along with farms as an income for a church or religious order. Unfortunately, town bathhouses are much more well documented due to them appearing in wills, in court cases, sales and rent purchases and letters of permission. We know much less about the rural ones.

From the Medieval County laws we also know that while not all might have their own bathhouse, peasant and their families probably still had access to one. How to pay restitution if you by accident burned down someone else's bathhouse is clearly noted down in the Västgöta law, for example. The religious section of the Uppland law also includes that among the buildings that are to be erected by the peasants of the parish for their priest is a bathhouse. It is quite possible that several peasant families shared bathhouses - they were often built close to streams or lakes, both for access to fresh water and for the ability to do laundry or just go skinny-dipping instead of pouring water over yourself at the end of the bath.

What we think of as a bath - submerging most of your body in warm water in a tub - was rare due to the large amount of work it required. Still, using wooden brewing tubs after the christmas beer had been brewed to take a bath - with the water shared among the family members - right before christmas seem to have been a common enough tradition.

Otherwise, washing your face and hands and combing your hair and beard (if you had one) every day are attested even before christianisation of Sweden. And skinnydipping in lakes or streams in Summer was also very common.

From at least the 1190s onwards, coroners in England were mandated to keep records of all sudden or unexpected deaths, from murders to accidents. From those records which survive, a common cause of accidental death throughout the medieval period is drowning in rivers while washing either oneself or laundry. This doesn't mean that hygiene was a death-trap, but does suggest - as you suggested - that visiting the nearest water source was the most common means of working on one's hygiene in the period for a great number of rural peasants. After all, a majority of settlements were built adjacent to or at least near rivers. Rivers can be dangerous, especially if you slip while wearing woolen clothing, or if you wade in to fast-flowing cold water while naked to wash yourself, and the periodic rate of accidental deaths suggest that this was a fairly commonplace activity for many people. After all, nearly 5,500 people drowned in domestic bathtubs in 2018 in Japan alone. Images such as these from a Book of Hours known as the Armorium Codicum Bibliophorum illustrate people bathing in water sources as a matter of course.

Bathing fully may have been a weekly occurence for most people, with simply washing using a bucket and cloth a far more common everyday activity.