Before unification, many didn't even share a common language. A person from Hamburg, who would have spoken a Low German dialect, similar to Dutch or even English, would have had (and still would have) a very hard time communicating with a Swiss German in his native dialect.

This is actually quite a complicated question to answer. Ethnolinguistically speaking, all Germans, as well as the Dutch, are descended from a single group of people that lived in Northern Germany around 2000 years ago, who in theory all spoke the same language which we shall call Proto-West-Germanic. This group of people, however, later split into several different tribes each with their own dialects, some that stayed roughly in their original homeland, such as the Saxons and Frisians, and others that migrated away, such as the Franks, the Alemanns and the Thuringians. Most of these tribes later reunited as part of the Holy Roman Empire, which can be thought of as a precursor to a united Germany, although was more of a confederation of many different smaller states than an actual bona fide empire.

During the latter half of the Middle Ages, these Germanic tribes started moving eastwards into formerly Slavic lands and settling there. However, many of the people originally living there, such as the Old Prussians who are most closely related to Lithuanians or Latvians, also became 'Germanised' and adopted the language and culture of their German overlords.