Edit: If there's any further reading on this subject, I would love it if you could direct me to it :)
The history of the Roman Empire can broadly be divided in two periods: there is the Roman Republic (~509 to 27 BC) and the Roman Empire (27 BC to ~476 AD [in the West]). This is sometimes confusing because 'Roman Empire' can be used for the entire period of 10 centuries as well as for just the 5 centuries in which Rome had emperors.
Anyway, I make this distinction because I'd argue that while Rome's conquests didn't contribute that much to the downfall of the Empire, they did contribute a whole lot to the downfall of the Republic and the transition to the Empire proper. Most of Rome's conquests happened in the period of the Republic; during the early Empire there were a few more (relatively small) additions to the empire but for the last 3 centuries of the Roman Empire there were hardly any.
Now why do I say Rome's conquests led to the downfall of the Republic? It is because the way the Republic was organised, and the way the Romans tried to prevent individuals from gaining too much power. The Romans abhorred autocrats. They had rid themselves of their king in the 6th century BC and now wanted to share power among the elite. They did this by electing two consuls every year. The consuls both had imperium - the power to command armies and to act in the public interest (a power originally belonging to the king). The power of the consuls was kept in check because each could veto the other, and because after a single year they had to make place for two new consuls. This way no one person could ever gain too much power.
At first everything went pretty fine. The consuls commanded the armies and led the state. Slowly Rome started to gain influence in Italy, and eventually came to dominate the entire peninsula. This was no problem in the Roman system, as everything was close by and every year the consuls could gather an army, march out, fight a war, and return home in time for harvest. Perhaps they had a bit too little time for their judicial duties at home, but that was fixed by instituting the office of praetor. A praetor also had imperium, but of a lesser grade than the consuls. From the 4th century onwards, every year there were two praetors elected to take care of judicial matters.
In the third century BC Rome faced Carthage and started expanding its empire overseas. Two long wars with extended campaigns stretched the Roman system to its limits. Extraordinary commands are given out during the wars to face the threats, and new permanent positions are created. After the First Punic War two extra praetors were elected yearly to govern Sicily and Sardinia. After the Second Punic War another two praetors were added to the roster (for six total) to govern Spain. Furthermore, the Romans came up with another idea to govern their expanding empire and fight their various wars. It was this idea that would prove to be disastrous.
At some point it was no longer feasible for Roman armies to return home every year. At first the Romans sent new commanders to an army every year, while keeping the army in the field, but that too proved to be impractical. So they came up with the idea of extending the imperium of a commander (consul or praetor) for a year. New consuls would still be elected, but the old ones could stay on as proconsul ("in place of a consul") and continue whatever they were doing. With this system of yearly renewal of imperium, the Romans started to chip away at the limits on power of individuals.
The Roman conquests had another consequence: slaves. Lots of slaves. Slaves who, among other things, could do the work of farmers. At the same time, those farmers who had a farm were often away from their farm for years because of military service. The Roman elite consolidated land holdings and replaced citizen farmers with slaves. The citizens flocked to Rome, hoping to make a living there. Summarising lots of complicated events in a short sentence: the Roman 'militia' army of farmers was soon replaced by a professional army with soldiers serving decades in the army instead of just a few years.
By now we're in the first century BC. Roman commanders lead, by necessity, armies for many years in a row. Roman soldiers serve, by necessity, under the same commander for many years in a row. What you have here is exactly what the Romans had tried to prevent from happening when they created their political system centuries earlier: single persons wielding tremendous power. The only thing which can stop a commander with a army loyal to him personally, is another commander with a loyal army. The commanders, always looking to expand their personal power, came into direct conflict and civil wars erupted. This lasted until one came out on top and everyone else had lost. Almost a century of civil wars ended with Octavian, the adoptive son of Julius Caesar, securing sole supreme power. The Republic had fallen because the Romans couldn't govern their conquests without giving individuals too much power.
I am less knowledgeable about the Empire than I am about the Republic, but I'll share a few thoughts which are related to your question. As I said at the beginning, there were very little conquests during the Empire. This is because the emperors had more or less the same problem as the Republic: they could not give one person too much power because that person would then become a challenger for the supreme power. This was less of a problem under the Principate (the early Empire) and certainly Octavian/Augustus had reliable lieutenants, but for many emperors it was the case that only they themselves could lead an army large enough for significant conquests. And if they would lead such an army, they'd be away from Rome for many years and give opponents there every opportunity to scheme. So emperors faced much the same problems as the Republican elite, and indeed there have been quite some Roman commanders who set up rival empires or even ended up becoming emperor of the entire Roman empire. In short, Roman emperors were very much concerned with internal matters and less so with conquests.
A lot can be said about the eventual downfall of the Western Roman Empire in the 5th century AD, but I'll leave that to those who have studied that period extensively. You can also have a look at the FAQ regarding that.
Further reading:
If you're looking for books, I don't really have any on this subject as it is quite broad. There are loads of interesting books about parts of what I wrote above. If there's anything in particular you want to read more about let me know and I'll try to find a good book related to that.
In addition to these very informative answers, there is also the cultural impacts that the Empire received through its expansion. Modern scholars put across theories such as "Romanisation" and "globalisation" which discuss the cultural exchanges between the Romans and there peripheral colonies, its reasonable (and evident) to assume that over time the idea of a "single roman culture" began to become diluted. This is seen in Roman Britain with the introduction of Celtic gods into Roman Mythology as well as in Egypt and the near east where similar phenomenons such as the idea of kingship occurred. This idea of absorption continued up until the 2th-5th centuries CE with the adoption of Christianity and the establishment of Constantinople as the "Roman capital". A good question to ask would be "when did the Roman Empire stop becoming Roman, so when did the "Roman" empire truly end ?" Was this change inevitable and to how much of an extent was the meeting of new cultures and religions responsible ?
I recommend reading Globalisation and the Roman World (2014) as well as the Fontana History books on the Republic and Later Roman Empire, both are very informative reads and the latter is much more easier to read and provides a great summary of Rome's two distinct time periods.