There's actually a term for what ancient people used to refer to themselves: an autonym (or endonym, but endonyms usually refer to places rather than languages or ethnic identities. For the purposes of this discussion, I will use autonym, but the two are basically interchangeable).

We know what many ancient groups called themselves thanks to writing.

As for Ancient Egypt, they called their land "Kemet" meaning the Black Land (referring to the extremely fertile Nile floodplain soil), and to themselves as Remetch en Kemet "People of the Black Land". Later, as society changed, the polity of Ancient Egypt was changed to "Hwt-ka-Ptah" (Ht-ka-Ptah, or Hout-ak Ptah), meaning "House (or Temple) of the Ka of Ptah".

(see http://www.touregypt.net/featurestories/kmt.htm)

As for the ancient Sumerians, according to W. Hallo, W. Simpson (1971). The Ancient Near East. New York: Harcourt, Brace, Jovanovich. p. 28., they referred to themselves as "the black-headed people" (ùĝ saĝ gíg-ga) and to their land as ki-en-gi(-r) meaning "place of the noble lords".

If an ancient group had writing, we are able to sometimes know what they called themselves, and sometimes what they called other people. For some of our modern names for ancient people, they derive from autonyms for those people or sometimes from xenonyms (names given to them by other people) if we don't have their name from writing. For example, we don't know what the ancient Philistines called themselves, because we have not been able to decipher the proto-writing symbols that they used. However, they were referred to by the Egyptians and others in the Medinet Habu reliefs and the Amarna Letters, and we know that these people called them "Peleset"

Now, it's important to draw some distinctions here. Ancient people groups were of many different types. You had seminomadic groups who had an ethnonym, but not really a ton of cities, or at least a name that others called them by (such as the Peleset). You also had polities known as City-States, which were basically a single large city which ruled over a hinterland of surrounding villages which supported it, and may have paid tribute to another group. This other group might be a massive empire such as Egypt, or it might be a nearby stronger City-State. Each of these city-states had their own identity, and a person from one of these would be identified by his city's name, as opposed to an ethnonym like the Sumerians' "black headed people" or the Egyptians' "people of the Black Land". You also had larger empires dominating and uniting a lot of cities which were wrapped up and assimilated into the empire's ethnic identity, such as with Egypt and Sumer, and all the other empires.

Ever since people began living together in cities, and interacting with other cities, they needed ways to identify specific cities and their inhabitants. In some cases, groups of cities teamed up into kingdoms and empires and needed a way to refer to the whole team, which led to the empire names. But originally, you had individual cities providing the identity and making it derive from the fact that the city is the uniting factor.