Ancient people in the Greek and Roman world did lift weights to build their strength, especially in the cities. However, wrestling was more popular and ball-based exercise was seen as superior (which I suppose can count as lifting weight since it is a weight and you lift it, but I assume you are talking about dumbbells and specialised gym equipment). I've included two classical weight lifting routines in detail, quoted at the bottom.

The idea of a gym is not a modern one, the ancient Greeks had the gymnasion, which is where the modern word comes from. Many ancient cultures such as the Egyptians and the Japanese practised wrestling, but we do not know whether they did structured weight training. We do have a lot of evidence for the Greeks though, who did.

The gymnasion was a a multi-purpose facility for the general improvement of one's health. Initially they existed to train athletes for the Olympics and other religious sporting events, and became popular once people quickly realised that working out made you more likely to win. We do not know when the gymnasia became a formal and regular part of public life in ancient Greece, but the writer Pausanias claimed that the Athenian gymnasia had been around for as long as Athens had existed. The politician Solon (638-558 BC) passed legislation governing the gymnasia, so they were embedded in public life from at least that point (A comment below by /u/Iphikrates has informed me that we need to take attribution to Solon with a pinch of salt even though Aeschines, Plutarch, and Demosthenes attribute the first laws about gymnasia to Solon). Physical training took placed under the supervision of the gymnastai, who were the ancient equivalent of fitness trainers. Most exercise concerned wrestling, with exercises such as one where someone would attempt to restrain someone trying to escape their grip, but there were ones which used equipment such as discus exercises. We do not know the exact equipment commonly used, but 8-lb dumbbells were initially used to assist athletes in the long jump and became part of strength building since they were convenient, and other weights and heavy objects were also used.

Plato states that the philosopher Prodicus was the first to formally note down the health benefits of regular exercise in the 5th century BC, though the benefits were surely known before he wrote them down. Over time, the gymnasia came to be about far more than weights and athletics and by the time of Plato included education and philosophy as well as building muscle - Plato's Academy was originally a gymnasion called 'Academy', one of Athens' three great public gymnasia (the others being Lyceum and Kynosarges). The writings of ancient Greek authors make it fairly clear that by the time of Plato, normal people were expected to attend the gymnasia for both intellectual and physical exercise.

We also know that many Roman villas began to include exercise rooms as of the 1st century BC, as the architect Varro exalted farmers for being healthy whilst 'not requiring the citified gymnasia of the Greeks.' and complained that 'In these days one such gymnasium is hardly enough, and they do not think they have a real villa unless it rings with many resounding Greek names.'

In the 2nd century AD, the Roman medic Galen wrote about the benefits of exercise, but did not like weights because he thought they only exercised one part of the body at a time. He did not like athletes because they had a habit of exercising to the point of injury. He wrote an entire treatise called 'On Exercises with a Small Ball' where he advocated using a small heavy ball to build muscle mass through a training regime devised to exercise the whole body - 'I particularly commend exercise which provides sufficient health for the body, concord of the parts, and excellence of the soul—all things which exist with the small ball.' He advocated this instead of running, dumbbells or push-ups and sit-ups because he thought they were boring, repetitive, and made people's minds 'lazy, sluggish and slow'. He also advocated ball-based exercise because they encouraged dexterity and hand eye coordination as well as strength, as well as being a bit more fun (in his opinion), and for meditative qualities - his routine included many exercises similar to modern yoga.

That being said, Galen did recognise the value in lifting weights and gives us detail of one popular ancient exercise:

"Some people place jumping weights in front of them that are six feet apart, then standing in between them and bending forward, raise them up, the one on the left with the right hand and the one on the right with the left hand, and replace each again in its proper place; and they do this in sequence repeatedly, keeping their base still. This movement particularly works out the lateral parts of the spine, just as the previously mentioned movement does those parts that are central."

He also wrote about a slightly more confusing exercise where:

"taking hold of some weight with each of the outstretched hands apart, such as the weights that are in the wrestling school, he holds the hands motionless, having extended them forward or raised them up. If he then were to direct someone to draw them down or bend them forcibly, while keeping himself immobile and rigid, not with his hands only but also with his legs and spine, he will be carrying out a substantial exercise"