This answer will not exactly be historical, but more from a polisci standpoint. Please let me know if that is not allowed and I will remove it.

For the Empire to run, whatever one it is and wherever, it needs money or some other form of wealth. To get money the government of said empire must either raid/pillage or implement taxes. Both have been used over time, but this question is referring to more “settled” empires. From a stability standpoint, taxes are an obvious way to get the most money for the least fuss. In order to implement these taxes and collect them, nations must have both some kind of method of collection and a force, or, “monopoly on violence”, which will, let’s say, encourage the people to contribute.

A method of collection would be who does the collection. Is it government ministers, local ones, a private enterprise, or something else? All are ways that various empires over time have collected taxes. The United States uses its IRS, or Internal Revenue Service, to directly collect taxes from every American via a central agency which is an arm of the federal government. The Persian Empire had its Satrapies in which regional Satraps, the governors of the day, would collect taxes from the people and then send the money up to the Shah. Rome on the other hand used a system of private companies which bid on contracts to exclusive right to collect taxes in a given area during the time of Augustus. The first two are obviously far more common methods, both which lead to a strong bureaucracy being developed to administer it. I won’t comment further on the IRS one way or another to not taint this with modern opinions, but the method of local governmental bodies collecting taxes which they then send to their superior can easily start to bloat because of ministers skimming off the top.

The necessity for force, even if it is simply a show of force, can easily get bloated as well. Depending on the time period and location, the necessary budget for the military can go through the roof. With a higher budget comes both an increased overall size as well as internal military politics. This can lead to it being ineffective and if it gets to a certain point, the people could even begin to refuse to pay up because there simply is little threat anymore. So the bureaucracy must be maintained in order to keep things going.

It isn’t that empires are unique, but more simply their sheer size that leads to bureaucratic bloat. Lots of land to cover and lots of men needed to do it. And more to keep everything in order. The origin of bureaucracy comes from the state needing to keep its affairs in order. Normally the most important affairs of a government is money. Not to say that all bureaucracy is tax related, but the same principles apply. They want to better facilitate things and setting up a bureaucratic system seems to be the obvious solution. Over time they tend to (not always, mind you) increase in complexity and size, leading to the reputation of empires for having a large bureaucracy.