It's hard to boil down a medievalists reputation to a single point, many individuals will have many reasons for disagreeing with or disliking someone. For many people, however, the controversy around Norman Cantor is related to his book Inventing the Middle Ages: The Lives, Works, and Ideas of the Great Medievalists of the Twentieth Century. While well regarded by some, it is highly controversial in some areas, particularly in how Cantor is sometimes less than kind (or even polite) to some of the luminaries of his own field. There are quite a number of reviews of the book available, and if you're interested I recommend digging some of them out (many are in academic journals, though, and may have restricted access). A free sample that includes most of a review from the American Historical Review can be reached here. I would classify this review as one of the most positive I've encountered, but it I think this passage from it summarizes the controversy around the book nicely:

Sometimes his judgments seem quixotic or lacking in generosity of spirit, as in his implication that Marc Bloch, who was shot by the Nazis in 1944, had a greater impact dead than alive. There is also a sense that in some case Cantor has sacrificed accuracy for a remarkably good story. Thus, he brands the Jewish emigre Ernst Knatorowicz a "Nazi," primarily on the evidence of his Kaiser Friedrich der Zweite (1927)

...

Although Friedrich II enjoyed such popular acclaim that even Kantorowicz later acknowledged that it helped pave the way for National Socialism, and although Heinrich Himmler kept the book on his night table, Cantor's account is difficult to reconcile with the classroom attacks Kantorowicz suffered from Nazi thugs at Frankfurt, the fact that his mother died at Theresienstadt, and his opposition to the loyal oath requirement at Berkeley during the McCarthy era.

Keep in mind that the above quote is from a positive review, so try and imagine how people who were less willing to overlook things like calling a famous Jewish historian whose mother was killed in the Holocaust a Nazi reacted to it. There are obviously other controversies around Cantor, any historian with his output is liable to encounter some, but a lot of the controversy around him stems from reactions to this book.

As an aside, the only Cantor book I tried reading was In the Wake of the Plague, and I found it to be poorly researched, needlessly provocative, and of a quality I expect from the most mediocre pop history rather than from a noted medievalist. I abandoned it after about 50 pages. This is not to say that all of Cantor's work is substandard, or that it has no value, it's more to say that I haven't been impressed by what I've read and it hasn't encouraged me to read more of his work. It's also worth noting that both of the books I've mentioned here are from near the end of his career, and Cantor would hardly be the first academic to go a little weird towards the end...