This is an absolutely fascinating question and one that I could discuss at some length. However, to summarize things in a cogent way, most doctors belonged to a guild that was often under the rule of a larger city council. Thanks to Gianna Pomata's Contracting a Cure, we know quite a bit about this structure in late medieval Bologna.

Basically, most qualified physicians were trained directly at the college of medicine in Bologna and would be licensed in some sense by the protomedici, the medical council. The protomedici more or less made decisions as to who was competent and who wasn't. Now, these physicians often were a rather insular group and medicine tended to run in families. When the physicians took on a patient, they often signed a contract that promised a cure and delivered payment up front.

Now, when these physicians (and as time went on, surgeon-barbers and pharmacists) failed to deliver a cure, they were often taken to court for breach of contract, which was a version of malpractice. These cases were, believe it or not, incredibly common. Pomata's book lists quite a few of them, and many of them are somewhat humorous. If the client could establish that a contract had been made, and that the physician/other attendant had failed to honor it, he often won. These trials were, in and of themselves, rather elaborate affairs, often calling upon the medical expertise of physicians, but also local people who had a limited knowledge of medicine. See J. Andrew Mendelsohn and Annemarie Kinelzbach's article "Common Knowledge: Bodies, Evidence, and Expertise in Early Modern Germany," ISIS 108, no. 2 (June, 2017), 259-279.

This system of a cure contract started to disappear in the early modern era, as expectations between patients and physicians changed from "I'll pay you if you heal me" to "I treat you and hopefully you get better."

In terms of determining who was competent and who wasn't, if you were lucky, your physician had been trained at a university under a competent medical staff. Other physicians had been trained in an apprenticeship and had learned their profession in that way. Generally speaking, however, poorer doctors tended to be placed into unpleasant roles (second-rate physicians were frequently plague doctors) and superior ones would minister to the upper classes.