I've been reading lately on the historical treatments of bacterial infections, and there's an article that addresses this exact question! I'll be pulling most of this response from that paper by J. Curtis Nickel.

We've got a roughly 5000-year history on the treatment of bacterial UTIs prior to the advent of modern antibiotics, and the contents of the works constituting that history range from lists of herbal remedies to lists of signs and symptoms of UTIs, as well as preventative and pain management strategies. The Pen Ts'ao is one of the first of such documents, and it contains a list of well over 1,000 herbs and medicines that could be used to treat infections, bacterial or otherwise -- and it was published over 5,000 years ago! One key example of treatments noted by Nickel is as follows:

Gravel and urine retention were treated with magnesium silicate and gokharu, hematuria was treated with white peony root, burnet and thuya or polygala infusions and rice and spiders were prescribed for frequent urination. Acupuncture was also practiced for many types of ailments, including urinary ones.

Interestingly, a number of these herbs have medicinal uses that have been demonstrated by modern scientific practices (e.g., white peony root, a.k.a. bai shao), while others have been shown not to have effects that have been traditionally ascribed to them (e.g., thuya/thuja).

Shortly after the publication of the Pen Ts'ao, the Nei Ching, one of the oldest medical books known, was also released. It largely dealt with how the pulse could be used for disease prognosis and understanding. There are also papyri from Ancient Egypt indicating herbal solutions to infections (Desner, E. - The history of urology to the latter half of the nineteenth century), though, as noted in the paper I'm basing this answer on, that discussion is minimal.

Examples also exist from Ancient Greece and Rome of specific treatments for urinary tract illnesses, both those we now know are caused by infections as well as things like kidney stones. Specifically, the collected works of Hippocrates (after whom the Hippocratic Oath, commonly taken by physicians, is named) include descriptions of kidney disease, including some that may be associated with UTIs:

The first of these descriptions most likely corresponds to renal calculous disease with or without infection and it is caused by phlegm accumulation in the kidney, which results in the formation of stones. The description of the second disease traditionally was believed to resemble renal vein thrombosis but it could represent many other renal conditions, including chronic renal infection...

Hippocrates recommended various treatments, the invasiveness of which correlated with the severity of the disease in the relevant patient. Those treatments included herbal laxatives (e.g., scammony, a plant in the morning glory family), kidney and abdominal surgery, pus drainage, and a diet seemingly designed to be both bland and likely to assist in producing bowel movements. As Nickel and others note, Hippocrates was one of the first physicians to incorporate evidence-based medicine into practice.

There were numerous other works over the ensuing millennia, but their contents were largely iterations on herbal remedies, descriptive documents that discussed what UTIs looked like but not how to treat them, and other papers/documents that sometimes discounted the value of even considering the question you pose here (e.g., Lanfranc de Milan, c.a. 1200). Other treatments were proposed, too, including enemas and bleeding, also during the 13th century. None of them notably improved upon the practices from Greece and China noted above.

Between the 13th century and the onset of contemporary antibiotic treatment of UTIs in the early/mid-20th century, prophylactic strategies like "better" hygiene and dieting -- in addition to regimens of herbs and chemicals, including opium, mostly for pain management -- began surfacing in the literature. Not until the work of Pasteur, Lister, and Koch did we see even hints of germ theory and microbial references in medical literature, which, of course, makes sense -- that's when germ theory began gaining widespread acceptance.