The evolution of federal law enforcement is largely the result of a longstanding wariness of the idea of a national police force. This is partly a function of federalism — state and local governments have long held primary responsibility for most law enforcement — but it also stems from a concern that a federal police force might evolve into a secret police organisation that could threaten the constitutional order and civil liberties. As a result, federal law enforcement agencies tend to be specialised, with relatively narrow mandates.

In fact, for most of the US' history, the broadest law enforcement authority was likely held by the US Marshals — but they were organised and administered at the district court level, and didn't have a centralised organisational structure until the creation of the US Marshals Service in 1969.

Most federal law enforcement agencies were been created in direct response to contemporary issues and crises: the Secret Service, for example, was established in 1865 under the Department of the Treasury to combat currency counterfeiting, because an estimated third of all currency in circulation immediately after the Civil War was counterfeit (which represented a major threat to US economic stability.) The establishment of the DEA in 1973 was part of Nixon's 'war on drugs', in response to rising levels of drug abuse and growing public concern about drugs during the 1960s. The ATF has its roots in the Bureau of Internal Revenue's enforcement units, and the Bureau of Prohibition created to enforce the Volstead Act; it gained its firearms mandate with the passage of the Gun Control Act in 1968 — a law whose passage was in large part a response to the Kennedy assassination.

The FBI has long been the most controversial federal law enforcement agency, because it's the closest to a true national police force. Created in 1908 to combat crime that crossed state lines, or that fell under federal jurisdiction but outside the direct purview of an existing agency, the Bureau of Investigation (as it was known until 1933) was immediately met with suspicion. During the First Red Scare (1918-20), the Bureau turned its attention to zealously targeting those it deemed radicals and subversives; its General Intelligence Division, led by a young bureaucrat named J. Edgar Hoover, conducted aggressive surveillance and harassment of suspected communists and anarchists.

In 1924, when the Coolidge administration came to power, Attorney General Harlan Fiske Stone set to reforming the Bureau — in part because of the excesses of the Red Scare years, but also as part of an attempt to clean up the Department of Justice after the notoriously corrupt Harding administration. Stone immediately curbed the Bureau's mandate, restricting it to strictly criminal enforcement (as opposed to political policing or counterintelligence) and telling the press:

"A secret police may become a menace to free government [...] because it carries with it the possibility of abuses of power which are not always quickly apprehended or understood."

(Quoted from The Boss: J. Edgar Hoover and the Great American Inquisition)

Thanks to a burgeoning reputation as an able administrator and arch-technocrat — and helped by his predecessor's implication in Harding-era corruption — Hoover was named director of BOI. Thanks to Hoover's gift for political manoeuvring and the BOI's high-profile involvement in pursuing celebrity criminals like John Dillinger, the Bureau was expanded and renamed the FBI in 1935.

Over the next 37 years, Hoover continued to enlarge the Bureau and broaden its scope — and in the process, became one of American history's most sinister figures and realised some of the worst fears of civil libertarians about a federal police force — most notoriously, with the COINTELPRO campaign of the 1950s and 1960s. In 1975, the Church Committee, investigating abuses by US intelligence and law enforcement agencies, quoted Stone's prediction in its final report, adding:

Our investigation has confirmed that warning. We have seen segments of our Government, in their attitudes and action, adopt tactics unworthy of a democracy, and occasionally reminiscent of the tactics of totalitarian regimes. We have seen a consistent pattern in which programs initiated with limited goals, such as preventing criminal violence or identifying foreign spies, were expanded to what witnesses characterized as "vacuum cleaners", sweeping in information about lawful activities of American citizens.

In the 1990s, following a series of scandals in the ATF and DEA, Vice-President Al Gore proposed merging both agencies into the FBI, but institutional interests and public opposition stalled the idea. Post-9/11, there was significant administrative reorganisation of federal law enforcement, with many agencies (the Secret Service, Customs and Border Protection, US Coast Guard) reshuffled to fall under the authority of the new Department of Homeland Security, but there have not been serious efforts to merge agencies themselves — likely because there's no political will or pressing operational reason to do so.

Edit: typos