...and caused so much destruction that the division was forced to withdraw

The British withdrawal which ended the movement phase of the Normandy Battle was kinda a result of the Villers Bocage battle but was a strategic decision which considered a lot of factors. If you just check the battle and the involved units you can see that the scope of the battle was rather small. A single tank commander can't cause the withdrawl of an entire division. The British decision is heavily debated in histography and the Villers Bocage situation seems to have had a overproportional impact on this decision considering the scope which gives some credit to the idea that the single tank commander Wittmann had big impact. The withdrawal is likely more a result of the lost momentum and the arrival of new German formations which now started coordinated defense and concerned the British commanders which showed over the entire campaign that they avoided to take risks. The 101st heavy tank battalion ( Wittmanns unit ) is an example for the now arriving German reinforcements.

Was Wittman just that much of a badass that he and 3 other tanks could cause such destruction and chaos among the Brits?

First of all the narrative of this battle was inflated by the Germans which were in dire need of a good propaganda story to asure the people that the new front was handled properly. Wittmanns tank count was inflated and the "success" was entirely attributed to him while he was just a part of the German force present. Nevertheless the informal title "badass" could seem justified not because of Villers Bocage but because of a long history of successful duty which made Wittmann one of the most prominent tank commander of the German Army. Quite ironically Wittmann got most of its critism for his actions during the Normandy campaign. He's often described as reckless ( Wolfgang Schneider ) which maybe cost him his life in august. Wittmann was used to fight Russian anti tank weapons and T-34 ( not T34-85) which might resulted in overconfidence in his vehicle. Its possible that a Firefly destroyed his Tiger and killed him and its crew. Regardless, the action of Villers Bocage was impressive. Wittmann destroyed a lot of enemy vehicles including up to 11 tanks and basically wrecked havoc. The narrative of a Tiger driving up against an superior enemy and causing destruction is powerful but not unique.

Was Michael Wittman's daring attack on Villers-Bocage really as incredible

Actions with such lopsided tank casualties were not rare for Tiger units. Like said previously it was exaggerated because it was the first battle of this kind on the newly opened Western Front. Since their first real combat use during Zitadelle ( mid 43) the Tigers proved to be far superior in duel situations which together with superb training and highly experienced crews led to extensive casualties of the Russian tank forces. Franz Staudegger is credited with the destruction of 20 enemy tanks during the Battle of Kursk resulting in the first Knights Cross of the Iron Cross for a Tiger Commander. A lot of comparable actions took place on the Eastern Front. Wittmanns actions at Villers Bocage, while impressive, were not unique.