First you have to take into account that the bodies represented by antique sculptures (1) are representations of an ideal and not the physique of your average ancient greek.

Then, the Greeks (at least the wealthy citizens of the city states) did exercise. Training at the gymnasium was an important part of the citizens life because

1- Aesthetics were considered the outside manifestation of an inside virtu and the sign of a strong will.

2- Physical exercise was a part of the military duty incumbent upon each citizen, as it was required to fight in an hoplitic formation.

3- The gymnasium was a place of physical training, but also socialization and learning. Working out was a part of the basic citizen social life.