It's pretty well known that during the Classical period, Greece was by and large divided into city states, which numbered ~700 at the time of the Persian invasion IIRC. Did the cities themselves comprise most of the Greek population at the time, or did most people live outside the cities in small, rural villages, like most agrarian civilizations prior to the Industrial Revolution? Do we know how the Greek countryside was peopled in Antiquity?
Classical Greek political life was, by and large, governed by the city-state system (there were some kingdoms, and some other such, but most "Greeks" in the Classical period lived in city-states). There were, of course, a lot of variations on the theme, but in general, a typical Classical Greek city-state was comprised of an urbanized center (an astu) and a surrounding "territory" controlled by the government of that center, a chora. The size of the urban center vs the chora could vary, with some states (like Athens or Corinth) controlling relatively huge territories, while others got a bit swallowed up and hemmed in (like Megara, lying between Corinth and Athens).
Belonging to a polis of some sort, be it a democracy like Athens, a narrow oligarchy like Sparta, a federation like at Thebes, or something in between, was a central tenet of what it meant to be a Greek. Not belonging to a polis was very precarious, and very dangerous. The polis provided a person with their rights, their identity, their security, and the backing for their claims on property. When one isolated hamlet of Attica was threatened, that hamlet could count on the full force of Athens to be behind them. In turn, it wasn't possible for an isolated hamlet to be "independent" of the astu--they were in the territory claimed by Athens, the chora, and that hamlet was Athenian whether they liked it or not. In most places, however, these identities were already coalescing in the "Dark Ages" and were already well-established by the Classical period. Independence movements among villages in the chora were not a thing, though sometimes settlements on the borders of rival poleis got to be quite confused as to whose chora they occupied. The process by which areas came together as city-states is called synoikismos, and it was a naturally progressing process through time which turned smaller communities into larger ones. Athens, by the middle of the 5th century, controlled as her chora all of Attica, a huge area compared to many other city-states. Numerous smaller poleis were absorbed in the process of forming that area. The Classical period is mostly dominated by these super-city-states, whose borders had slowly (or abruptly, as the case may be) expanded into each other, gobbling up less advantaged communities along the way. Those early poleis lost their political institutions and became participants in the larger state (except in the case of Sparta, where they became helots).
If you could transport yourself to a random farming hamlet in the early 5th century BCE in, let's say, the northeast of the Peloponnese, you could make some quick observations and very quickly discern that hamlet's "affiliation," so to speak. You could ask a farmer, "Where do you go to vote (if you vote)? Where do you go to honor the gods? At what sanctuaries? With whom do you fight? What does your pottery look like? What do your weapons look like, your armor?" From these answers, you could get a quick but fairly accurate understanding of to which polis that hamlet belonged. The world of the gods, in particular, played a fundamental role in contributing identity to the chora. In fact, it is widely accepted now that the very early connection between emerging city centres and peripheral sanctuaries served to delineate the chora of a city-state. This is the content of de Polignac's work (Cults, Territory, and the Origin of the Greek City-State 1995). The inhabitants of the astu and surrounding lands made regular trips to cult sites to honor this or that god, and by that action came to identify with one another as fellows. Their procession to the sanctuary, repetitively, served to "mark out" the space between as belonging to their particular collective. Argos and the Heraion, for instance, or Athens and the sanctuary of Poseidon at Sounion, or Athens and Eleusis.
For Aristotle, belonging to a polis was the quintessential expression of the Greek man's life. A man without a polis was inferior, even subhuman (Pol. 1253a). Above the polis level of organization was the ethnos, and below it civic identities (like your tribe, or deme), but these were never developed like polis identity. Xenophon considered the polis to be one of the essential aspects of civilized existence (Mem. 1.1.16).