The various Germanic groups of the late Roman Empire did not emerge from the primordial forests of Germania, entire civilization in tow, to wrack a terrible destruction on the Romans for no other reason than their own inherent barbarism! Indeed relations between the two areas, Germanic realms, and the Roman world, were complex and varied a great deal over time.

These Germanic societies had long existed on the periphery of the Roman world and were heavily involved with the empire through trade, war, alliances, and all the other hallmarks of two neighbors. Roman trade goods and material culture have been found across Germania, even into Scandinavia.

The increasingly popular view of the migration is not that they were monolithic and homogenous distinct entities arriving in Roman territory for a new home. Indeed the actual number of migrants and their status, military, civilian, or a mix is still hotly contested.

In the case of Britannia, Robin Fleming's holds that the migration was indeed a largely peaceful process with little evidence of the endemic warfare that one might expect. Peter Heather argues that this migration events were primarily economic in nature, warlords and their retinues seeking to make their fortune inside the empire, either through raiding or by mercenary service, or migration.

Let's take a look at the Goths more specifically since you mention them in your post!

The Goths had led prosperous lives on the fringes of the Romans world until the depredations of the Huns forced them to flee their ravages and try to escape to the safety of Roman territory. The Goths split into Visigoths and Ostrogoths at this time, the Ostrogoths remaining under Hunnic authority for a time while the Visigoths crossed into Roman territory.

Traditionally, the Goths are marked as the epitome of the participants in the Voelkewanderung. An entire society uprooted by the onslaught of the Steppe peoples, who then poured over the Roman borders in their thousands bringing their entire populations in search for a new home. The Romans let in some of the refugees but not all, eventually enmities between the groups flaired up and the Goths were responsible for some of the worst disasters in Roman history, the battle of Adrianople and the 410 sack of Rome come to mind. Eventually the Visigoths settled in Aquitaine and Iberia and the Ostrogoths took over Italy.

From this traditional view we get a lot of the misconceptions that come down to the modern day.

This view has a LOT of flaws in it.

Any attempts to reconstruct the migratory past of a pre-literate society are always going to run into some large issues. There is archaeological evidence of a (partly) Germanic culture occupying space around the Black Sea that seems to have fallen apart alongside the rise of the Huns, the Chernyakhov culture. However any attempt to solidly connect this culture to the Goths is going to be tenuous. Efforts to construct a migratory history of the Goths based almost solely off of Roman accounts, which were often written well outside any realm of contemporary, are doomed to failure.

However the biggest issue with this traditional view, and this gets to the gist of your question, is that the Goths were perfectly willing to not only work alongside but fight for the Romans. They did not arrive on the Danube river looking for Roman blood and maraud the countryside until they got their fill of it. Indeed it is hotly debated just when (if ever) Gothic identity solidified around "visigoths" and "ostrogoths". Peter Heather argues that this process did not happen until the various groups that all comprised the forces under Alaric were in Roman territory.

Throughout their interactions with the Romans there were Gothic forces who clashed with the Romans then turned around and fought alongside them, only to then end up back at war with the Romans. The "visigoths" fit this to a T.

They first arrived, fleeing the Huns supposedly, in the late 300's and strike a deal with Emperor Valens for safe haven. They eventually rebelled due to poor treatment by the Romans and delivered several defeats to the Romans. Peace was eventually made and the Goths moved on through the Empire. They were then roused to violence once again after the Roman general Sticlho was executed and the Goths were targeted by the Romans. This is what prompted Alaric's attack on Rome.

Indeed following Alaric's sack of Rome the Visigoths established themselves in southern Gaul as foederati of the Romans. They participated in Roman wars against the other "migratory" groups at this time such as the Suebi, Huns, and Alans. Eventually this relationship broke down and the Visigoths carved out their own realm in southern Gaul and Iberia.

Now this timeline of events is a holdover of older scholarship, however there is little basis to overturn the basic order of these events. The big controversies largely stem from disputes over the number, ethnic/cultural make up, and composition (all military, civilians mixed in, mostly civilian, etc) of these groups.

This was not a question of the Romans being unable or unwilling to deal with the Goths on their territory. In fact they dealt with them as both allies and enemies quite extensively. Relations ranged from subservience, outright warfare, to alliance, and eventually the Ostrogoths preserved a host of Roman institutions in Italy.

Now, why didn't the Romans deal with them once and for all during one of the periods of warfare between the two groups?

Rome at this time was not the Rome of centuries prior. The Goths had achieved quite successful results against the Romans in the past as well. The infamous battle of Adrianople when the Goths managed to destroy an entire Roman army and kill the Emperor Valens comes to mind. In addition to this, the Goths were only one off and on enemy that the Romans had to deal with. The other groups, that the Goths helped fight as often as not, also engaged the attention of the Romans, not to mention the rival Sassanids, Civil Wars, and Imperial pretenders that so often plagued the late Empire.