In the Graeco-Roman world, alcohol was very widely used for recreational purposes, generally in the form of wine. Beer was known but not widely consumed; distillation of alcohol into spirits did not yet exist.
There is plenty of evidence for the medicinal use of opium and, to a much lesser extent, cannabis -- mentioned by Dioscorides and Pliny as primarily beneficial for treating earache. There is, however, essentially no evidence for recreational use of cannabis (plenty of mentions of its useful for making rope), and while Galen records that opium was overused by, for example, Marcus Aurelius, this need not be seen as recreational as opposed to the poorly understood effects of addiction following use for medical purposes. In particular, I do not believe that there is any evidence of smoking as a means of taking either drug; I believe that Booth's "Opium: A History" identifies smoking as a New World tradition that was brought back to Europe. There have been various proposals that psychedelic mushrooms were a part of religious ritual as far back as the Neanderthals, but the specifics are hard to pin down. Psychedelic mushrooms were certainly available in the Roman world. Psychedelic ergot doesn't lend itself well to recreational use, and was more endemic to the colder, moister climate of northern Europe, but it is possible that it was present in the classical world to some degree. Cocaine and its derivatives as well as synthetic drugs such as LSD would have been unknown in ancient Europe. Someone else may be better able to speak to recreational coca use in the Americas.
In other words, the evidence is not strong for a widespread culture of recreational drug use. Very interesting, however, is the story of the lotophagoi -- the lotus-eaters of Homer's Odyssey. Odysseus says of the lotus:
which was so delicious that those who ate of it left off caring about home, and did not even want to go back and say what had happened to them, but were for staying and munching lotus with the Lotus-eaters without thinking further of their return.
That could certainly have its roots in a recreational use of a plant with psychotropic properties, although this is not absolutely certain. There is considerable debate about what species of plant Homer is referring to here, and Homer's travel tales are well-mixed with bits of mythology, but Herodotus claims the lotophagoi still existed; he has little to say, however, about the lotus except that it was sweet like a date and the locals could turn it into wine.
In ancient Egypt, the Festival of the Return of the Distant Goddess celebrated the return of Sakhmet, the lion-headed destroyer goddess, who returned to being Bast, the cat-headed goddess of sexuality and other fun things. The narrative behind the festival was that when Sakhmet was tired from all her work destroying the enemies of Ra, someone - I don't recall who - gave her red-dyed beer to appease her bloodthirst. Thinking it was blood, she lapped it up, and then fell asleep, and when she awoke, she was Bast again.
The festival had to be celebrated outside the temple proper, basically on a giant porch, and celebrants had to drink wine and red beer to excess, and also took an intoxicant made from lotus flowers for ritual purposes. They were all in a pretty weird place when at the height of their drunken sexual orgy, the statue embodying the goddess would be brought forth among them.
Interestingly, the litany in several Books of the Dead known as the Negative Confessions, where you tell all the gods of judgement what evils you didn't do so they don't condemn you forever, you were supposed to confess to not having been given over to drunkenness. So while drunken, drugged sexual excess was a cultic staple in the worship of a major goddess, just kinda going crazy on your own terms may have been considered a sinful violation of social order.
TL;DR: Yes, ancient Egyptians had crazy drunken, drug-fuelled sexy orgies, but maybe only when it was appropriate to the religious occasion.
Well since OP didn't specify, how about in ancient Greece and the Roman Empire?
Where? When? "Ancient times" could refer to hundreds of different cultures. In comparison to which modern cultures? Are we comparing ancient China with modern Persia? Are we comparing ancient Rome with modern Amazonia?