Basically, no. While medieval people were generally very skeptical of women's ability to not be oversexed little lustdemons, the beliefs that (1) celibacy was basically impossible and (2) nuns were being horribly exploited by the Church, tricked into practicing empty ritual instead of focusing on real salvation, are both inventions of the early modern era. Nuns, like other late medieval religious, did indeed receive plenty of criticism. However, while we hear stories of "priestwhores" and "monkwhores" who specialized in a particular clientele of randy clerics, most of the attacks against nuns involve financial waste. (Don't get me wrong--there's plenty of that particular accusation to go around.) Discussions of nuns and illicit sex generally involve questions about an ex-nun fleeing her convent--breaking her vows--because she is pregnant or in order to get pregnant/start a family. But no, in the Middle Ages, "Get thee to a nunnery" was not a euphemism in the Shakespearean sense.

That said, there are a couple of complicating factors. First, particularly in urban Italy, religious foundations did operate houses for wayward girls and women. This could included former (or "former") prostitutes. These communities ran basically along monastic lines, including a reduced version of the daily Hours (schedule of communal prayer). They are categorically not brothels; however, you can see a clear link between them and a red-light former (and in some cases, probably future) life.

The second is the thorny--and debated in scholarship--question of whether convents, specifically in England, owned brothels or owned the property on which brothels operated. The scattershot nature of medieval legal records, the tendency of brothels to change names from tax period to tax period, and the ambiguities of location in London's Southwark district, have led many really excellent medievalists to identify the monastic community of Stratford-at-Bow as the owners of a brothel called the Unicorn ("and maybe others"). Henry Kelly has recently argued this is a misnomer, that the "Unicorn" owned by the nuns was not a brothel during the time it can conclusively be placed in their community registry of property. I'm not sure there is enough evidence to draw a conclusion either way--Kelly's argument hinges on a couple of major gaps in the source record, but so does the other view. It seems to be a case of reading into the holes the view that one already holds. (Ecclesiastical-related ownership of brothels in late medieval London is not out of the question; evidence from literary fiction suggests popular opinion may well have associated nuns with the operation of brothels if not participation themselves).

It's possible that future research could turn up a case of a convent that was running a prostitution ring with its own nuns, but I honestly doubt it. Medieval convents tended to be either reform-minded and very very dedicated to a spiritual life of prayer and contemplation, or sort of a prestigious alternate career to marriage and marriage alliance-making for noble (especially princely-level) daughters. Plus, for every anticlericalism-tinged complaint about nuns in furs eating meat every day, the vast majority of stories of convent life in the later Middle Ages turns on financial hardship, especially as stricter claustration rules prevented more and more women's communities from engaging in productive (money-earning) labor. There's not a lot of room for convent-brothels in this paradigm.