Scholars used to believe in a linear model of linguistic development in ancient Mesopotamia: The Sumerians migrated into the region, founded the first cities, and were conquered by Sargon and the Akkadians, who replaced the Sumerian language with their own, Semitic, language. Today we have a much more nuanced and complicated picture of that process.

There is evidence for the use of Akkadian and Sumerian from virtually all periods of cuneiform history, including the earliest periods. Names in Sumerian or Akkadian are short phrases, and as a result we can identify and analyze the grammar appearing in them. A name can contain Sumerian or Akkadian elements, and sometimes both, but our weaker understanding of the writing system concealed this nuance until a few decades ago. It seems to us now that Akkadian was spoken right alongside Sumerian, even before writing, especially in northern Babylonia.

From the perspective of history in the ancient world, the dominant language is the administrative language because those are the institutions that produce the documentation that we read. Cuneiform writing was invented to facilitate book-keeping and accounting, and scribes throughout the Ancient Near East were primarily employed to manage the financial systems of the period. So when someone says that Akkadian “replaced” Sumerian in ancient Mesopotamia, they mean that the language of administrative documents changed.

The administration of the Sumerian city-states changed c. 2334 BCE when Sargon’s forces conquered the whole of the region. Part of his new governmental policy was the appointment of both state and temple officials who were loyal to the regime, and it seems that these officials had a preference for Akkadian over Sumerian. The same scribes likely were doing the same work under the new regime, but they likely shifted the language to accommodate their new bosses. We do not have any inscriptions talking about the new king’s linguistic policy, so it may have simply been the personal preference of the new officials just as it could have been a result of a royal policy.

The use of Akkadian spread rapidly beginning in this period, and it will eventually turn up in Syria, where it replaces Eblaite, and will go on to eventually appear in Egypt and Anatolia. Sumerian will return as the administrative language during the Ur III period (c. 2112 – 2004 BCE) before declining, but it will remain a spoken language for some time and literary language for the rest of Babylonian history. In fact, the very last cuneiform tablets ever written were still primarily using Sumerian.

For more info, see Michalowski 2000 - "The Life and Death of the Sumerian Language in Comparative Perspective" for overview and bibliography