Were they even very effective?
Well, it depends on when and where you're talking about. I would suggest that the Classical Era,circa 600 BCE- 500 CE. Now to deal with the where.
Let us, right off the bat, talk about India. Indian Princes would frequently and generously use war elephants, mainly as platforms for archers, but also as heavy assault forces. There are accounts of the most powerful princes having twenty thousand or more elephants in their military. This is where elephantine warfare became prevalent for the first time, and continued to be through the entire classical time period. It is through this route that elephant warfare first came to the Near East, and the West, excepting the aberrant occurrence of Carthage slightly later on.
After the death of Alexander the Great, there is a period of warring generals, known as the Diadochi. During this period, one of the leading generals, Seleucus, spent a small amount of time fighting near the Indus River Valley. The newly formed Mauryan Empire was able to stop any possible Greek expansion, and eventually, the rulers came to a deal. Seleucus would leave India, and the Indus River valley on the whole, to the Mauryan Empire. In return, he was to be given 500 war elephants. This demonstrates the importance of the novelty.
His use war elephants is by far the most effective, and the most prolific. He brings his 500 elephants to bear in the Battle of Ipsus, which pitted Seleucus, Lysimachus, and Cassander against Antigonus. While Antigonus had a small number of elephants, through the former armies of Alexander, Seleucus had such a large number that he was able to overwhelm with them, and use them tactically to cut off enemies as needed.. Aside frm this battle, many Hellenistic states bred elephants, but the use of elephants was not particularly prevalent in the Greek states, and never again was it anywhere near as effective.
One other, very popular example deserves examination, though it is more an aberration than the normal, just as elephants use overall. The Carthaginians developed a type of elephant warfare, though it is best attested through Hannibal, who lost more of his elephants in his trip through the Alps. These elephants were a bit smaller than the most notable elephant today, the African Bush Elephant. Though they used them frequently, they did not play as large a part in Carthaginian military power. Hope this is helpful.