This is not a particularly easy question to answer. Barbaric populations haven't left any significant account of how their groups were organized or what kind of factors contributed to the sentiment of homogeneity in said groups.

Nonetheless, we can count on archeological findings, classic written sources, later written legislations from the Roman-germanic kingdoms as well as later written narratives from authors belonging to those populations after their settlement in the ancient Roman Empire (calling them "same populations" would be indeed inaccurate) and philological analysis.

The most probable answer is that those groups were, contrary to Tacitus and ancient Romans vision of their neighbors, heterogenous. Archeological evidences provided a variable amount of clear answers. Most settlements don't correspond to a defined ethnic unity, while in some cases (as for the Avars) the population seems to present a more recognizable cultural profile.

Another important aspect is that that the same criteria that can be used to identify an ethnic group living today or even three hundred years ago cannot be applied to European late antiquity.

Wenskus theory was that those populations weren't a "racially" defined unity, but rather groups of people that self-identified within a common ancestry. This "faith" was guaranteed through myths of origin and common customs and laws. An example of a origo gentis myth is from Paul the Deacon on his Historia Longobardorum (VIII century), where he recounts the origin of his people, their way out from Scandinavia and how they were favored by Freyia and Wodan.

Nonetheless, these groups lumped together by core traditions lived in an extremely dynamic environment and, as a consequence, ethnic identities were naturally dynamic themselves. The VI century Goths were probably a very different population from the ones that were labeled ad Goths two centuries before.

Ultimately, it seems like barbarian populations were mostly organized in small tribes, where only the elites of warriors, and possibly other prestigious figures as clergymen had a larger area of influence. Populations knew they shared a common matrix only under specific circumstances and only under certain identifying categories.

I've heard that some of these groups were actually tribal confederations.

This is a good way to think of them, at least at first. Ethnicity is fluid and often in a state of flux. Many smaller tribes would frequently band together to treat with or fight the Roman Empire. Small tribal identities would give way to the identity of the confederacy. Those confederacies would then become Romanized to varying degrees, either through friendly contact or conquest, while also having their own influence on the Romans. The exact nature of these groups is often a matter of debate. Particularly when discussing the Goths, Gothic ethnogenesis is a hot topic.

The oldschool point of view was that the Goths represented a homogeneous ethnic group that migrated from Scandinavia into what is now Poland, then later migrating into Romania/Ukraine/Crimea (a region Jordanes calls Oium). The more modern stance is, at least in my opinion, unnecessarily reactionary- You'll hear the Goths called 'heterogenous multiethnic polities', or some variation of that phrase, indicating that there were many different non-Germanic ethnicities joining onto a small number of East Germanic elite, the warrior aristocracy. The Scandinavian origin is usually downplayed or outright rejected. Their ethnogenesis is now placed in Poland or the trans-Danubian region, depending on how early you'd like to place it and what you would consider a decisive ethnogenesis, despite evidence for various tribal ethnonyms being carried over from Scandinavia.

The reality is likely somewhere between the two perspectives. There was certainly contact across the Baltic, though there's no evidence for mass migration. Proponents of the multiethnic polity perspective often cite Slavic groups as likely latchers-on, though there's little evidence for this. There's only a single Proto-Slavic loanword in the Gothic corpus (plinsjan, 'to dance' from Proto-Slavic *plęsati). East Germanic has left many more loans in Slavic, likely well after Wulfila's translation of the Bible via East Germanic speakers who didn't enter the Roman Empire with the Ostrogoths and Visigoths. We see the occasional Hunnic and Iranian names occurring among the Goths in Greek and Latin sources, but no Slavic names. By and large, most names are East Germanic, and most loanwords are from Latin and Greek. Grave sites associated with the Ostrogoths show occasional occurrences of cranial deformation, a practice borrowed from the Huns. I believe the number was around 5%, though I can't find the source on that. On the other hand, we don't see any Hunnic loanwords in the Gothic corpus, and the attested names are hardly representative of the population as a whole. Members of the elite are more likely to be mentioned by name in sources than commoners.

An important point to keep in mind is that ethnicity is fluid- a family of Huns or early Slavs might only take a generation or two to become fully Gothicized. Either Gothic group would likely have a portion of its population at various stages of Gothicization, from families who have become fully enculturated to the point that they may no longer be aware that they are not direct descendants of any migrants from Scandinavia, to those that are recent latchers-on who may not even speak much Gothic yet. Both groups included wagon trains of families and were hardly composed of only fighting men, as is sometimes claimed to support the multi-ethnic polity point of view, as if they were simply very large bands of mercenaries gathering around the core East Germanic warrior aristocracy. Both groups were also undergoing a process of Romanization, with the Visigoths at least a generation or two ahead of the Ostrogoths in the adoption of Latin. Whether or not the Visigoths even spoke Gothic by the time they settled in Hispania is debated among Gothicists, while the Ostrogoths certainly used the language in Italy (we have land deeds from Naples and Arezzo in Gothic). Another key point is that the far side of the Danube, where the Chernyakov archeological culture was situated, seems to have been largely (but not entirely) East Germanic in the 4th and 5th centuries. The traditional (oldschool) perspective is that the Tervingi became the Visigoths (or Vesi) and the Greuthungi became the Ostrogoths. The reality is that the Ostrogoths and Visigoths, as we understand them today, were polities of mostly East Germanic, or recently Germanicized, peoples who reformed into those respective groups. Peter Heather breaks down their makeup like this:

Visigoths:

1 the greater part of the Tervingi

2 the Greuthungi under Ermenaric

3 Goths led by Radagaisus

Ostrogoths:

4 Amal-led Goths

5 Goths under Theodoric Strabo

Others:

6 the remaining Tervingi, perhaps the same as those led by Arimer

7 Greuthungi led by Farnobius

8 Greuthungi led by Odotheus

9 Goths under Bigelis

10 Goths under Dengizich

11 Crimean Goths

12 Goths near the Sea of Azov

The likely scenario is that small groups of early East Germanic speakers brought ethnonyms into Eastern Europe- Goths, Rugii, Vandals. They then incorporated other local non-East Germanic peoples and spread into the trans-Danubian region before being pushed into Roman territory by the Huns, where we see them reforming as the Ostrogoths and Visigoths, among others. Both the Romans and Huns played a pivotal role in shaping the Ostrogothic and Visigothic polities. The Visigoths entered the empire earlier, escaping Hunnic subjugation, while the Ostrogoths and Gepids broke off from the horde after the Battle of Nedao. There were certainly Goths that did not join either group, notably the Crimean Goths and Wulfila's Goths in Moesia.

In Italy, the distinction between Goth and Roman was not only ethnic, it was also legal. Goths and Romans were subject to different sets of laws. While not officially allowed, intermarriage certainly occurred. The Ostrogothic kingdom didn't last long enough for the legal distinction to entirely supersede the ethnic distinction, though it did so in Visigothic Spain. The Visigoths primarily spoke Latin, and the survival of Gothic is a touchy subject. As wealthy landowners, who would have been considered Romans, took Gothic names and intermarried with Goths, Goth came to mean a member of nobility.