I hear people use the names Babylon, Assyria, Sumer, and Mesopotamia pretty much interchangeably on the internet, were they different cultures, the same culture, or different versions/ages of the same culture sort of like new/middle/old kingdom in Egypt?

This is understandably confusing. There's a lot of overlap in these names and a lot of different historical uses.

Mesopotamia was the Roman/ ~~Latin~~ Greek name for the area between the Tigris and Euphrates rivers (and the surrounding area. The same word is used today as a catchall for the same area in all of ancient history. There was no individual Mesopotamian culture, but there was a general sense of Mesopotamian cultural continuity over almost 4000 years, which brings me to:

Sumer, which was the first major "civilization" in the area, and arguably the world beginning around 3500 BCE. Civilization meaning agriculture, urbanization, social hierarchies, and maybe writing depending on who you ask. Geographically, Sumer covers the area south of modern Baghdad (more or less). It's also probably the most definitively distinct culture on account of its language and religion. The Sumerian language is a language isolate, meaning it is structurally unconnected to all other known languages/language families. Naturally, this meant that names, including the names of their gods, were derived from Sumerian.

Around 2300 BCE, there was a bit of a cultural shift as a new element was forcibly injected into Sumerian life: the Akkadians. Akkad was a city somewhere in the vicinity of modern Baghdad (specific location still unknown), and home to Sargon of Akkad, the founder of history's first true empire (according to most definitions at least). Akkad and the surrounding area north of Sumer were culturally different from the Sumerians. Akkadian was a Semitic language distantly related to the likes of Hebrew and Arabic, and with that came a different pantheon of gods from the Sumerians.

During the time of the Akkadian Empire, there was something of a cultural merger. Mesopotamia became largely bilingual with Akkadian and Sumerian being used in different contexts. Their religions also started to mix, with some Sumerian and Akkadian gods remaining seperate, while other Sumerian deities were conflated with similar Akkadian gods. In the latter scenario, aspects of both original traditions mixed together.

Sumer held out as a distinct cultural block for a few more centuries, but was gradually supplanted by a more Akkadian/Semitic dominated culture with Sumerian elements. The Sumerian language remained in use for religious and literary purposes all the way to c.100 CE, but in a context more like Latin in the modern world. Through that time there was a brief Sumerian renaissance called the Third Dynasty of Ur around 2100-2000 BCE, but the arrival of more Semitic cultural groups like the Amorites gradually led to those elements taking precedence.

While all of this was happening, Assyria was just kind of doing its own thing. The first evidence of Assyrian culture at their original capital of Assur comes from around 2600 BCE. Geographically, the Assyrians were concentrated in northern Mesopotamia, specifically the region between Assur and Nineveh along the Tigris River. Most of Assyria could be described as modern Iraqi Kurdistan.

If they were ever linguistically distinct, we don't have evidence for it. By all indications they were culturally/religiously in line with other Semitic peoples and started using an Assyrian dialect of Akkadian early on. Technically, Assyria was conquered by both Akkad and Third Dynasty Ur, but they weren't really big players yet. The Assyrians only emerged as a signficant political and economic power in the 18th Century BCE. Even then, they were still largely confined to their section of northern Mesopotamia.

The 18th Century BCE is also when we get into Babylon. After the collapse of Third Dynasty Ur, Mesopotamia was dominated by different hegemonic city states. You can kinda-sorta compare the situation to Classical Greece on larger geographic scale. The northern Tigris River had Assyria. The middle Tigris (basically modern Diyala, Iraq) had a place called Eshnunna. The geographic area of Sumer was contested back and forth between the cities of Isin and Larsa. These places were all culturally and linguistically pretty similar. There were local variations, but the differences were largely political and economic.

Then there were the outside players, like the kingdom of Mari based in Syria and Elam in southwestern Iran. They're important because a series of Elamite invasions of Mesopotamia allowed the king of a little city called Babylon, named Hammurabi, to build his own power base. Hammurabi, famous for issuing a very well preserved law code later in his life, got his start making strategic conquests and smart alliances until he was able to conquer most of Mesopotamia. That was from around 1792-1750 BCE.

He was never able to conquer Assyria outright, but he did force them to submit and pay him tribute. That arrangement basically led to the political situation of Mesopotamia for the next 1200 years. Assyria and Babylon both had periods of ascendancy and periods of defeat, but for the most part, Babylonia was solidified as the area from the Persian Gulf to somewhere just north of modern Baghdad and Assyria was the region of Mesopotamia north of that.

Culturally, religiously, and linguistically they were very similar with room for some notable local flavor. Akkadain remained the lingua franca of the whole region, and was used for international politics and trade from Egypt to Iran because of the influence of those two Mesopotamian powers. Outside powers briefly conquered both regions at multiple points, and during the height of the Neo-Assyrian Empire from c. 813-651 BCE, Assyria functionally ruled Babylon. In 612 BCE an alliance of Babylonians and Medes conquered Assyrian and destroyed its capital at Nineveh. After that point, Assyria was never a political power again, but none of these events ever led Babylon and Assyria to lose their distinct local cultures despite their shared Mesopotamian milieu.

How both names were used changed after Mesopotamia came under functionally permanent foreign after the Neo-Babylonian empire was conquered by Persia in 539 BCE. Over time, especially in the periods where historians are forced to rely more heavily on relatively distant Greco-Roman sources, both "Assyria" and "Babylon" were still used, but there is less consistency. Sometimes they're synonymous, sometimes they're distinct. "Assyria" in particular shifted in geographical meaning a few times.

I glossed over a lot of history here just trying to explain the names, but if its a time and place that interests you A History of the Ancient Near East, ca. 3000–323 BC by Marc van de Mieroop is relatively up to date and a relatively easy read.

Hello! Concise answer first: Babylon, Assyria and Sumer were different things, all within Mesopotamia, and they each have their own chronology. Mesopotamia is a greek name meaning "between rivers". It refers to the area between rivers Tigris and Euphrates, a fertile land in which many different peoples thrived since Pre-History.

So onto the Sumerians, who are the most ancient of these three. We have evidence of them from the end of the Neolithic and beginning of Bronze Age. The Sumerians' origin is still disputed, but they possibly moved to southern Mesopotamia from the East. They spoke Sumerian, an isolate language: one for which no relatives are yet known. This is one of the reasons why it has been difficult to estimate their origins. They created one of the first writing systems known, the Cuneiform script, at first for accounting, but also for literature and other types of texts.

The Sumerians were followed by the Akkadians - you might have heard of Sargon of Akkad, the empire's founder. Akkadians were a Semitic people and spoke... Akkadian! Their territory was significantly larger than the one occupied by the Sumerians, and they're often called the first world empire. They adopted Cuneiform and many other elements from the Sumerians.

After the collapse of the Akkadian Empire, the Gutian rule, and a new Sumerian period, emerged the states of Assyria and Babylon, in, respectively, North (city of Assur) and South (city of Babylon) Mesopotamia. Assyrians and Babylonians both spoke Akkadian, each with their own dialects, while still using Sumerian in written language. Both have their own chronologies, and in some periods, one city ruled over the other.

At the 6th century BCE, the Neo-Babylonian state, which had succeeded in ruling both Assyria and Babylon, was conquered by the Achaemenid Persians.

If you'd like to learn more, you can check out the World History Encyclopedia, easily accessible online. (I can't post the link, but you can find it easily on Google) As for books, I find Mario Liverani's The Ancient Near East: History, Society and Economy to be a great resource for learning about the history of the region as a whole. It's quite big, though.

Experienced historians, if I got something wrong, please tell me - I'm still an Assyriologist-in-training :)