The Classical Greeks considered themselves united in part by their language, and it seems they could all understand each other, but there were a couple of distinct dialect groups. Wikipedia has a lovely map showing which areas used which dialect. The main ones were:
Ionic, spoken by the Ionian Greeks who inhabited the islands of the Aegean and the cities on the coast of Asia Minor (modern-day southwest Turkey)
Doric, spoken by the Greeks of the Peloponnese (including the Spartans, the Argives and the Corinthians) and Crete, as well as their colonists overseas (including Syracuse and other Sicilian cities)
Aeolic, spoken by the Greeks of Aeolia (now northwest Turkey), Boiotia and Thessaly
Arkadian, spoken by the Greeks of Arkadia, a mountainous part of the Peloponnese, as well as the Greeks of Cyprus
Attic, spoken by the Athenians
Each of these dialects had its own peculiar features, as we can tell from texts written in one dialect or another. Since the vast majority of surviving Classical literature was written in Attic Greek, students learning ancient Greek in school or university nowadays are actually taught Attic. However, some words and names are still more commonly known in their dialect form - like the Doric "Sparta" instead of the Attic "SpartĂȘ".
After the conquest of Greece by Macedon, the Macedonians encouraged the use of a new common dialect or trade language called KoinĂȘ Greek, which was based on Attic.