The thing that started urbanism (people settling in cities) was the rise of agriculture, a phenomenon known as the Neolithic Revolution.

Prior to the rise of agriculture, people subsisted on hunting and gathering, which requires people to be nomadic, following the food, and requires them to be in relatively small groups. It has been estimated that hunter-gatherer groups require between 18 and 1300 square kilometers per person, depending on the terrain and available food resources. Therefore, even a small group would require a large area. Hunter-gathering as a subsistence method results in small, spread-out groups migrating over large tracts of land. Obviously, this means that cities are not a logical, productive, or useful means of settlement in this sort of economy.

But then, people figured out how to domesticate their food sources, enabling them to create surpluses of food. This took place in the Neolithic period, roughly 10000 years ago. V. Gordon Childe was the first to propose the link between the development of agriculture and the beginnings of urbanism, and you can read his work online: Childe, V. Gordon. "The urban revolution." Town Planning Review 21.1 (1950): 3 (PDF Warning). The main idea is that food surpluses allowed for non-food-producing specialists to be part of a group of people. Fewer people could feed more people, and this allowed food-producing specialists to provide for craft and other specialists, such as craftsmen, chiefs, bureaucrats, scientists and priests, to create and innovate such important technological advances as writing, metallurgy, city planning and scientific principles (Weisdorf, J. L. (2005), From Foraging To Farming: Explaining The Neolithic Revolution. Journal of Economic Surveys, 19: 561–586) (PDF Warning).

Another advantage of cities, once agriculture was developed, was the ease of defense. Because once people settled down and began creating things that might be considered “wealth” (such as food surplus or luxury goods innovated as a result of having a food surplus to support specialists), then people began trying to steal that “wealth”, and having a larger group of people for defense of that collective wealth was beneficial. So people settled together to protect and feed their assets, and built walls.

Therefore, cities arose because agriculture arose and made them possible. Prior to agriculture, it was not beneficial, logical, or desirable to have permanent settlements, let alone cities. After the development of agriculture, it became essential. This is why cities arose after 10,000 BC.

Now, your next question will be “why did agriculture only get developed after 10,000 BC?” /u/Mictlantecuhtli explains below that there were major climatic issues that prevented reliable, consistent agricultural production from being developed, due to the Ice Age and other major climatic events. It takes an ideal environment for agricultural technology to develop, usually near rivers. This is why the earliest sedentary civilizations cropped up near rivers (Egypt and the Nile, Mesopotamia and the Tigris/Euphrates), because the environment was extremely fertile and well-suited to producing the most food naturally, as well as being very climatically stable, since the rivers provided a constant source of water.

I believe it was a combination of poor climate opportunities as well as the idea just took a long time to occur. Change happened very slowly in hunter-gatherer societies. Even tool innovations took millennia to develop, allowing us to date groups based on their tool styles. I believe it was mainly because every person was part of the food-procurement effort, and nobody was able to devote themselves solely to innovation, therefore innovation took longer.