Here's an older answer from u/Celebreth on why axes were not used by the Romans at their height. The mechanics of using a war hammer are basically the same and can be ruled out for the same reasons.

In fact, the explanation is basically the same for the Greeks if you adjust a little bit for the different style of fighting. In Greek (Archaic, Classical, or Hellenistic) warfare, the spear was king, and swords were generally a sidearm. Like the Roman legion, Greek warfare was orchestrated around a tight formation - the phalanx - which was usually even less flexible than their later Roman counterparts. It had to be, given the overall length of their weapon of choice, there just wasn't much room for movement. On top of that, the defensive structure of the Greek phalanx was premised on overlapping shields that could not be turned aside for a sweeping blow with a hammer or ax, just like Celebreth described in relation to the Roman scutum.

It's not that axes and hammers did not come about in warfare. They were just poorly suited to Greek and Roman formations. Other cultures that fought in less tightly grouped or shielded formations absolutely made use of those weapons for different purposes - even earlier Greek culture. The labrys was a type of double-bitted ax that appeared in artwork, especially religious or ritual art, all over the Aegean, and functional versions of both the labrys and other designs have been found in Bronze Age contexts predating the Classical cultures we're talking about here. The Romans describe various "barbarian" groups as using axes in battle. Both Greek and Persian art depict the Scythians and Persians using the sagaris a long handled ax with a small blade and spike or hammer on the other side that probably developed as a cavalry weapon.

Cavalry was probably the motivating factor that eventually lead to the reintroduction axes and hammers in Roman warfare. Zosimus described Palestinian auxiliaries wielding maces and clubs successfully against the Palmyrene cavalry in 272 CE and the development of heavily armored cataphracts on both the Roman and Iranian (Parthian and Persia) sides of the Euphrates led to the adoption of maces and hammers as cavalry tools. As I noted above, the sagaris had been a longstanding weapon of Iranian cavalry since the 5th Century BCE, but by the early medieval period both Persian and Byzantine sources were referencing bludgeons as regular parts of cavalry warfare, sometimes in discussion of the early Late Antique period.