Taking a look at the Wikipedia list of law enforcement agencies in different countries, I noticed that the list for the US is a lot longer and more comprehensive than for other countries. While this might be a result of a US-centric design of the article, from my personal knowledge of my own country (Germany) the difference in number still stands out.
In the US almost every federal government department seems to have its own agency, while in Germany and most european countries the number seems to be much lower with a few central police services and no departmental services.
On a state level this seems even more the case, though with a larger country that is also less centralized this isn't as unexpected.
I'd assume that on a federal level this is largely a result of a larger country as well plus the pure additional age the USA has over Germany, the modern german state being less than 100 years old and therefore less bloat developing so to speak. But are those the only reasons?
Someone else asked the same question last week — here's my answer.
tl;dr — a combination of federalism making people generally mistrustful of the idea of a true federal police force, and a gradual but massive expansion of the range of crimes that fall under federal rather than state or local jurisdiction.
There are many focus-and-efficieny reasons that are at the bottom of some. As well as concern for overloading centralized, administrative, technical, or practical resource services. Things so simple as managing multiple specialized applications or meeting differing federal requirements pertaining to specific tasks (healthcare privacy, confidentiality as relates to legal matters, and the different levels of required education and training to handle varied fields).
However, one wrinkle that ends up creating some that I've personally witnessed working in government is the influence of political or campaign-related objectives. An agency gains the reputation as too bloated so they'll split part of it off into a separate agency or give a program to a different agency to make the former's budget look smaller. Someone will campaign on a specific healthcare, military, or economic issue and then have to find a way to either create a new agency/program or peel off and inflate part of an existing one. They're hard to fully eliminate once you have legislation propping them up and we end up with an alphabet soup. Sometimes for the better, sometimes the other direction.