During the late Republic and early Empire as far as I know plenty of recruits still came from Italy and other provinces that had been incorporated in the empire for quite some time. Why and how did this change to the point where the army was so dominated by Germans, both within the rank and file to high level commanders of dubious loyalty, that they could seriously threaten the Imperial government? What was the cause of Romans avoiding military service during this time period when in earlier centuries they were much more likely to join up?
The third-century crisis posed massive threats to Roman towns on the fringe of the Empire. Civil war and usurpations had sapped a huge amount of soldiers from border forts and manned defences along the frontier, so it became increasingly difficult to meet basic military needs. This led to a system of elastic defence (adopted out of necessity) where Roman armies were required to defeat barbarians inside Roman land. Why am I describing the turbulent and destructive period of the third century? Well, the practical side of it is that soldiers were being killed at an alarming rate, and not being replaced in a fast enough manner to keep everything running properly. Secondly, imagine yourself as a Roman citizen living in southern Italy during this time. Let’s say your father, his father, and maybe even his father all served in Roman armies, and you are expected to follow their lead. Hearing about how dire the situation must have been, would have really wanted to give up your comfortable Mediterranean life to risk being put into a frontier army? Probably not. An unwillingness to join up at this militarily and politically dismal time contributed to the spiralling problems in the army, specifically those within recruitment.
Roman emperors of the period (as numerous as they were) were presented with the problem of maintaining the defence of such a vast area (that stretched across Europe along the Rhine and Danube rivers, and constituted the entire eastern front with Persia) while being limited by a lack of actual people to serve in the armies. The answer was to conscript everyone, everywhere, including Germans. This was not an unusual approach as ‘barbarians’ had served in legions throughout Rome’s history, though never in the numbers they did in the Late Empire. The ‘barbarians’ themselves would have had massive incentives to take up the offer of recruitment; they would eventually be able to retire as citizens within the Empire, and on a practical level, they lived in these border towns either within or just outside the Empire, so they were defending their own homes more than anything. Additionally, a ‘barbarian’ officer class became more prominent and so it would not have been untoward to have ambitions of an esteemed military career; (though much later) Stilicho (himself a ‘barbarian’) became the power behind the emperor’s throne and the Emperors Valentinian and Valens were ‘beer-swilling’ Pannonians. As well as this, emperors might not have seen growing ‘barbarian’ numbers in their armies as a threat, as you say, but actually as an advantage. During a time where emperors were consistently overthrown and unsure whether their secretaries would murder them over a minor dispute (as was the fate of Aurelian), it might have been attractive to have a ‘barbarian’ bodyguard due to the knowledge that they could not usurp due to their non-Roman identity (namely that the vast majority of soldiers were unlikely to proclaim a ‘barbarian’ as their emperor - during the third century, that is). Diocletian was fond of these battle-hardened soldiers and often had ‘barbarian’ regiments attached to his armies.
Overall, it seems that it was rather a good solution to the problems facing the Empire at the time, and if the approach was not adopted the borders may have been overrun to the point of collapse. The question one might ask is why this wasn’t adopted during earlier crises of conscription, but it seems that the third century specifically was such a prolonged disaster that earlier debates about the dilution of fighting quality must have been dismissed to allow the widespread adoption of barbarian regiments into the Roman army which, funnily enough, was unlikely to have reduced the actual skill of Roman armies (later defeats and mutinous behaviour were undoubtedly due to many other debilitating factors present within the Empire, not the multiethnic makeup of individual Roman armies).
TL;DR, it is a case of necessity on the most part, with crises of conscription and civil wars there was no harm in having more men, especially ones who didn’t pose a direct threat to an emperor’s position.
Bibliography
Corcoran, S (1996), ‘Empire of the Tetrarchs’ (Oxford). Rees, R (2004), ‘Diocletian and the Tetrarchy’ (Edinburgh). Williams, S (1985), ‘Diocletian and the Roman Recovery’ (New York).