Lets skip the obvious "Alexander should have brought back elephants" but and talk more about Rome. Between the Punic wars and wars with Persia, they certainly knew they existed. Even after Roman collapse, medieval Europeans loved the image of Eleazer Avaran pulling a Leroy Jenkins on the Persian emperor's war elephant.
Why didn't the Europeans ever acquire a breeding population of elephants to use as labor and mounts? Every civilization that was known to the Romans used elephants, yet they did not.
Also, side question: lions seem absolutely inescapable in European heraldry despite being from Africa. Why were elephants not as popular.
Side question 2: did the Egyptians use elephants, and if not, why not? They're the closest to where to find them (out of the bronze age indo-mediterranean belt of civilizations), and they probably would have appreciated their size for building purposes. Besides, I've never seen an elephant represented in Egyptian art.
I think there are actually two issues that need to be fleshed out here. One is that elephants in general are not usually domesticated, so much as tamed. They breed slowly and live in fairly small groups spread over wide territories compared to, say, horses, cattle, or sheep. They also eat a LOT, and those factors militate against domestication in all but the most optimal circumstances. To my knowledge, outside of modern zoos, there have been no successful large-scale and long-term captive breeding populations. Instead, they are captured from the wild and put to work or war as humans can manage. As such, any society deploying elephants needs to be adjacent to an ecology that can sustain elephant populations. And, because elephants require such large amounts of food and space, you need a large amount of that habitat to sustain a population large enough to be useful. It's possible in parts of South and Southeast Asia, but even then it's not a given that a society will even want to use elephants, perhaps because they eat so much and are so difficult to manage, useful though they are. Mark Elvin argues, for example, that Chinese environmental history has been a long and one-sided contest between elephants and humans. Humans have been spreading agriculture (and later industry) at the expense of the forests and fields that sustain elephant populations for thousands of years. They have consistently driven elephants away, seeing them as pests that ruin crops. (See his book The Retreat of the Elephants)
The second issue is environmental. Elephants do inhabit a wide range of ecologies, as this map of the historic range of Asian elephants and [this map of the historic range of African elephants](http://magazine.africageographic.com/wp-content/uploads/2014/07/Hist-Africa-31-650x650.jpg illustrate. (Obviously, those are since the last ice age or even more recently.) But, while there are elephants that can inhabit either forests and grasslands, they do need fairly warm climates. As such, Europe north of the Alps is simply too cold for them. And, Mediterranean Europe doesn't seem to have had the wide open spaces necessary for large populations. Any elephants in Mediterranean Europe would be competing with humans for space, and by the time of the Roman empire the land is fairly well populated. There's simply no suitable reserve of tropical or sub-tropical forest and grassland nearby to sustain a large population from which humans could draw.
So, while it's really impossible to demonstrate why something has NOT happened, it seems pretty clear that elephants were just not available in large numbers in Europe.
The Romans did use elephants, and according to Polyaenus employed them as far as in the invasion of Briton.
"Caesar had one large elephant, which was equipped with armor and carried archers and slingers in its tower. When this unknown creature entered the river, the Britons and their horses fled and the Roman army crossed over"
Polyaenus, (VIII, 23.5)
They were also employed in Colosseum for entertainment, apparently, according to Seneca in De Brevitate Vitae .
Something to keep in mind, though, is that in areas outside of Africa, elephants are fairly difficult to maintain. Hannibal crossed the alps with like 20 elephants still alive, but after the Battle of Trebia, he's already lost all his elephants not thanks to the Roman Legion, but to the bitter attrition he suffered in northern Italy. So to employ them in the same scale seems difficult to the Romans, since a sizable chunk of their land might be deemed uninhabitable by elephants
Panthera leo europaea, the European lion, was only extinct in Greece in about 100 BC. It was the size of an African lion with a notable mane (unlike the earlier cave lion seen in cave art). They were found as far north as Germany. They lingered in Transcaucasia until the 10th C AD.
P. l. Persica wasn't called the Persian lion for nothing, though it now survivs barely in India, often called the Asiatic lion. There are so many bas-reliefs of hunts by Persian kings featuring lions. This breed was found as far west as Eastern Europe. They survived in Mesopotamia & Syria into the 1800's.
So no problem for Europeans to mythologize, see, record, and import lions down the ages. But if you see Durer's or Mantegna' drawings of elephants, you have to call them weird-looking.
For all that, Charlemagne's favorite pet was an elephant. But they didn't take to the climate, tropical beasts that they are.
