Actually, the Greeks would exclaim "By Zeus!" as it's often translated. Both the Greeks and the Romans would invoke a god or Herakles/Hercules. From my experience working with ancient texts, people would call upon Zeus/Jupiter more often. But it was possible to call upon Bacchus or Demeter or other deities when needing an exclamation. And the (dated) expression "By Jove!" comes directly from the Latin Iupiter, Iovis.

Such exclamations are really common in Aristophanes and other comedies which feature the 'every day' man, who is rather crude. Also, the topics and or humor tend be vulgar and thus they tend to 'take the lord's name in vain' fairly often. For example as we might curse, "Jesus Christ, you're stupid!" A Greek might exclaim: "By Zeus, you're ignorant! ἀμαθής γε νὴ Δί᾽!"

Yes! At least from Greece and Rome, oaths like this are preserved in literature that may have tried (consciously or otherwise) to imitate colloquial speech. Latin has "edepol" ("By Pollux!", attested in Terence) and "hercule[s]" or "meherc[u]le" ("Oh Hercules!", used even by Cicero), and Greek has "[ναὶ] μὰ Δία" ("[Yes,] by Zeus" - Socrates says this a lot in Plato's dialogues) and "[ναὶ] τὼ σιώ" ("[Yes,] by the Twins [Castor and Pollux]," in Aristophanes). The god invoked seems to have varied by location: Castor and Pollux had a large veneration at Sparta, and the oath from Aristophanes is in fact spoken by a Spartan character, though I'm not sure how realistic classicists think this portrayal is.

Is this something that is common to many cultures around the world is this more specific to the West?