The significance of the creation of the OSS (and subsequently, the CIA) was that it was the first time the US had a central, standalone intelligence agency. Prior to that point, the US was still involved in intelligence gathering, but on a much more fragmented basis.

Before 1942, the State Department, FBI, Army and Navy all had their own intelligence-gathering capabilities. The Office of Naval Intelligence was established in 1882; the Army had a long military intelligence heritage dating back to the Revolutionary War, becoming more formalised with the Civil War-era Bureau of Military Information and the establishment in 1885 of the Military Information Division. During the First World War, the Army created the US' first dedicated signals intelligence unit, MI-8, which was disbanded and re-established as the State Department's Cipher Bureau in 1919.

The creation of the OSS was presaged by the appointment in July 1941 of Bill Donovan as Coordinator of Information — a civilian chief of intelligence, attached to the White House and tasked with drawing together the disparate US intelligence community.

The major problem Donovan faced throughout the brief history of the COI and OSS was entrenched resistance from the military intelligence establishment, which was deeply skeptical of the idea of civilians muscling in on the espionage game:

Senior American military officers thought an independent civilian intelligence service, run by Donovan, with direct access to the president, would be "an extremely dangerous thing in a democracy," in the words of Major General Clayton Bissell, the assistant chief of staff for military intelligence.

— Tim Weiner, Legacy of Ashes (2007)

To some extent, those problems have never been resolved. The CIA remains just one component of a vast intelligence bureaucracy (today, the US Intelligence Community includes 17 agencies across six cabinet-level departments) and jurisdictional feuding has been a recurring feature of US intelligence history.