It's not "it's been said", it's that he specifically said these things himself in his own writings. His writings and paintings are well-documented, as are many of the events of his era.

As far as scholarship goes, I've read several books mentioning him and I think they were all from Shambala. It's worth noting that these aren't scholarly writings by academics, but writings from a different kind of scholarship, which is the budo world itself. On one hand, there is depth and understanding. On the other hand, the criteria for academic scholarship doesn't apply.

Whether the stories are embellished or not, lost in translation or oral transmission, people I've discussed it with (high-ranking martial artists) aren't too concerned with specific details being true or not because the points made are more esoteric. Book of Five Rings was his meditation as well. The stories mean the most as something you meditate on, not something I answer for you. The point is the process, not the conclusion (like any koan).

Yes. I have studied Miyamoto for years, purely for curiosity and this is what I have learned.

The stories (80%) have 3rd party witnesess. These duels were not just two guys walking past each other and then fought and nobody spoke about it (although musashi did wander through the woods for many years fighting wandering ronin). Musahi did use mental advantages against his opponent, this is not false.

He showed up late with a duel with a Master swordsman(kojiro). This is well documented. Because Kojiro was a by the book kind of guy, he knew it would get under his skin if Musashi showed up late. (2hrs)

Musahi was also known to throw his wakizashi, a move that was very unorthodox, but even in his youth, he was said to have practiced this move by throwing his wakizashi into trees.

He would not give his opponent time to think, or gather themselves. Once when the duel between Musashi and another swordsman was being discussed, and the time organized, Musahi stood up and screamed NOW, and pulled his sword to challenge the swordsman, he obliged, but was defeated.

I'm getting into a lot of fights in the comments here, but I guess I'll ask this as a root follow up.

What are the primary sources that says he won 50+ duels? Are there accounts of these duels, or a list of opponents or something?

Musashi had a lot of tricks that he would use against opponents.

He says to make sure that your opponents eyes are always looking into the sun. Is this cheating?

Then there's the concept of using the high ground. Oldest trick in the book. Is this cheating?

At a certain point, as long as the guy is winning a duel with a sword, it's pretty hard to call him a cheater. It's not like he's using poison or holding children hostage. He's winning by having an intimate understanding of what swordfighting actually is, and how to make it more difficult for the other guy. If the other guy has less sword-fighting knowledge (doesn't realize he is being forced into facing the sun) then he is losing the fight in another way, a sword-less way. You might want to read "The Life Giving Sword" for more thoughts on winning sword fights without actually using the sword.

By the logic of the OP, you could argue that Sun Tzu's "Art of War" is the biggest "cheater's manual" in known history. "Never let your opponent choose the terrain for a battle" is just cheating......right?!

Read Musashi's work....

Were these "duels" 1 on 1 fights to the death? Did he kill 50+ people or just make them submit?