I wrote about Alexander the Great's personal involvement in combat here. While Greek sources are not very interested in fighting ability, and Greeks were as likely to mock as praise someone for practicing weapon proficiency, they were very interested in leaders giving the right example by taking the same risks they force upon their troops. This was widely recognised as a key element in combat motivation and unit cohesion. As such, all Greek and Hellenistic commanders were experienced fighters and many of them carried the scars to prove it.

To the best of my knowledge nothing is said in the histories about Hannibal's personal fighting abilities.

Caesar must have fought in frontline combat during his early career as a junior officer in the legions, but not much is said about it apart from a line in Suetonius saying that he was decorated for bravery during the Siege of Myteline. By the time Caesar became a general he was in his early forties. He led troops personally when the situation called for it (such as at Alesia, where he led the relief force that plugged the gap in the Roman lines), but he would have been surrounded by his personal guard so it's unlikely the enemy ever got close enough to him for hand to hand combat (although at Alesia the fighting was pretty heavy, so maybe).

By Napoleon's time generals usually didn't take part in combat personally. Napoleon started his career as an artillery officer, so even before he was a general he didn't have many opportunities to get into close combat (although he was noted for his bravery under fire). That said, as a young captain he was injured during the Siege of Toulon when he led a successful counterattack against a British sally.

As for Alexander the Great, he was a maniac even by ancient standards. He didn't just go into battle with his men, he made a point of riding right at the tip of the formation. At Granicus he personally killed two of the enemy commanders. At Gaugamela he led his cavalry in a charge directly into the Persian centre, straight at the Persian king and his bodyguard (who fled rather than face him). Fighting at the front of the formation was common for Greek generals, but Alexander often fought ahead of the main formation. He would literally throw himself into the middle of the enemy and rely on his bodyguards to come to his rescue.

At one fortress in northern India Alexander's men weren't showing the proper enthusiasm (by this point they were tired of campaigning on the edge of the known world and would rather have just gone home). So he grabbed a siege ladder from one of his men, climbed up it alone, and then - in an example of what I will call 'Alexander logic' - because he was exposed and in view of enemy archers he jumped off the walls. Not off the walls back to his own men - off the walls into the fortress, right into the middle of the enemy. He then proceeded to cut down several enemies single handed while his men frantically tried to catch up to him.

The accounts we have probably aren't exaggerated, because not only do they describe him fighting, they describe him nearly getting himself killed through his recklessness. The histories describe an all too realistic list of injuries that Alexander suffered over the years. For example, at the aforementioned Indian fortress Alexander was shot in the lung and pelted with stones. He kept fighting, but soon passed out and only survived because a few of his bodyguard made it over the walls to him in time.

From all this we can deduce that not only was Alexander insanely brave, he was insanely good at fighting. You can put some of his success down to having the best armour money could buy, but if Alexander regularly behaved like this for over ten years worth of campaigning then the only way he could have survived is if he was a demigod in hand to hand combat.