The current responses have part of the answer, but are rather incomplete in that there are two aspects that need to be stressed: one the Roman Empire was not special. Contemporary with the Roman Empire, the Han Dynasty controlled an empire that was slightly bigger than Rome's and had, if anything, more difficult frontier problems. The Maurya and Gupta bookended Rome in India, and the Persians controlled significant chunks as well. All of these empires are different in systems of control and in ideology, but it goes to show that we should not find large, agrarian empires unusual. A second aspect is that Rome was very much a light touch empire, absolutely dependent on a degree of local consent. Simply put, Rome did not have the capability or the desire to manage affairs closely in its provinces, and by and large, as long as the tax system was maintained and the borders were secure, locals were left more to their own devices.

That being said, even though it had a light touch the presence of the Empire was still transformation. This is because the nature of Roman society and administration was fairly integrative, and the borders between Roman and non-Roman were fuzzy and highly permeable. This meant that the empire established a degree of persuasive normativity, by which I mean that it became "the way things are". This can most interestingly be seen in many of the revolts against Roman control: Julius Civilis and Zenobia, although very different in most respects, are similar in that the local administration was still essentially run along Roman lines, with consuls, curites, and other trappings of Roman rule. This gave the empire a fairly helpful degree of inertia in its running, which is always useful.

This normativity was established, as I noted, because of the permeable boundaries of Roman identity. As the Empire expanded, so too did the definition of "Roman". This went beyond simple awards of citizenship, and there were quite a few provincial senators who were not "Roman" of heritage. This essentially meant that local elite had a stake in the success of the Empire, as it provided channels to wealth, power and prestige unimaginable for their ancestors. This process of co-option is essential to all empires, but Rome was particularly good at it.

I think looking at the military and personal networks is very important, but the essential aspect of Roman administration is that it relied, to a high degree on qualified local consent. The Romans not only had to use the stick of military force, but the carrot of wealth and prestige that emphasized the benefits of the imperial system for those who controlled local power.

I have been recommending this a lot, but Greg Woolf's Rome: An Empire's Story is a great introductory text on these issues.

There were a number of tools a Roman governor and the Emperors had at their disposal for the administration of the provinces. These start coming more into effect during the Imperial period (previously, though this was not a general rule, being appointed a governor was considered a reward for Roman praetors and consuls, and they'd go there for a year and effectively swindle the provincials).

First and foremost is the military, the means by which they got these vast territories. Those provinces which were allotted legions in the Imperial period (Britain, Germany, Moesia, Africa, and Egypt) would use them for peacekeeping, the defence of the Empire's borders, and the control of trade traffic along those borders. Governors of provinces which didn't have their own legions could also apply to the Emperor to receive a detachment of troops from places that did (Pliny, for instance, once wrote to the Emperor because he wanted a detachment to protect trade in the town of Juliopolis).

The military also gave rise to colonies, urban centres which were founded to house Rome's veterans. These would be surrounded by farms which served as part of the veterans' retirement fund, and so would have the effect of producing more food and promoting Roman culture overseas. They would also be tasked with defending the Empire if necessary.

Second - you'll have heard of the term "bread and circuses"? To avoid the problems a hungry and dissatisfied civilian body could bring, it was common practice to give handouts of grain and hold games to keep citizens content. During games, a large number of people from many social classes would be gathered together, meaning the games' sponsor would get a lot of recognition, and citizens could, as they often did, petition a governor or Emperor directly if they were dissatisfied by a certain thing.

Then there was public building. If there is a way to make people know how generous, wealthy, and powerful you were, it was by erecting big buildings in your own honour. Emperors did this a lot, making "gifts" of massive buildings, to promote a feeling of inclusion, unity and oh-by-the-way-the-Emperor-can-crush-you-like-a-bug-ness. A good example is the Temple of the Severans at Djemila in Algeria, calculated to make the average citizen feel as humble as an easily-crushable bug. Likewise, gifts of aqueducts, sewage systems, etc. promoted the idea that Romans owed their life to the Emperor.

Also, roads. It's easy to take for granted just how vital these were for communication, administration, and ensuring the Empire was kept connected. The whole Empire was a network, with branches upon branches that ultimately led to one city. Take an educated guess as to where that city might have been.

An important tool, though we may not think of it as one, is the supposed client-patron system of Roman sociology (although client-patron systems have been identified outside of Rome also). A useful case study is that of Trajan and the younger Pliny, who came to be the provincial governer of Bithynia, since their (probably extensively edited and posthumously published) correspondence survives quite well in book 10 of Pliny's letters.

Rather than just being a faceless bureaucrat able to hide away inside the administration of the Roman dominion, Pliny had a personal relationship with Trajan prior to his appointment to the governor of Bithynia, in that Pliny was Trajan's client, meaning that Trajan could trust in his judgement. It's a little bit like the Mafia, with Pliny as a trusted member of Trajan's family of clients. Pliny likely won his spot by proving his capability and trustworthiness to Trajan by prior service, which gives ambitious Romans some direction in which to channel their ambition rather than leaving them languishing and 'champing at the bit'. This means of 'keeping Romans busy' seems to have been acted on at first by Sulla, who instituted the 'cursus honorum' during his 1st-century BCE dictatorship. This structure survived the Republic.

Pliny's correspondence reveals a relatively hands-off approach to governing the province, at the encouragement of Trajan. Of particular interest to many historians is Trajan's instruction not to extensively persecute the Christians (in letter 10.96), but other issues include the decision not to allow the formation of an association of firefighters in case it turns into a political club and of the locals wasting money on vanity projects.

These administrative networks are perhaps, surprisingly, very organic. Roman administration seems to have relied on informal relationships of patron and client, with the princeps (emperor) and governor at the top, but also between the governor and his various staff which many include skilled slaves and freedmen (freedmen were clients to their former master). At first glance one might expect such an informal network to be brittle and chaotic, but it seems to have withstood the test of time quite well.