There were really four different organizations in the FRG tasked with spying during the Cold War and in theory, each worked their own territory. The main German intelligence agency, Bundesnachrichtendienst (BND/Federal Intelligence Agency) was focused on conventional spywork, especially with maintaining contacts throughout Europe and was directly subordinate to the Chancellor's office. Domestic intelligence was the task of two agencies, the Bundesamt für Verfassungsschutz (BfV/Federal Office for the Protection of the Constitution) and its Land counterpart, Landesämter für Verfassungsschutz (LfV/ State Office for the Protection of the Constitution). In some ways the BfV operated in the mold of the FBI, investigating overarching domestic crimes, conducting counterintelligence work, and conducting surveillance and analysis of domestic problems. But there are major differences between the two organizations. Neither the BfV not LfV could legally make arrests or conduct interrogations, so they had to operate in a complex web of arrangements with Land police. Finally, the Bundeswehr maintained its own intelligence service, Militärischer Abschirmdienst (MAD/Military Counterintelligence Service) tasked with preventing sabotage and conducting counterintelligence, but also preventing anti-constitutional elements from taking root inside the FRG military.

This web of intelligence networks, interrelated, but bureaucratically separate was a byproduct of the experience of the Second World War and the arbitrary police networks of the Third Reich. There was a widespread consensus among the Western Allied governments and intelligence agencies that German intelligence should "be no new Gestapo," which was a sentiment echoed in Bonn among the Adenauer government. Thus FRG intelligence was to be a new agency for the new Germany. The BfV's name, for example, averred to the importance of the Basic Law and its agents were unarmed. The federal bans on both the KPD and the SRP were justified in part by evidence gathered by domestic agencies, showing the agencies were investigating both left- and right-wing antidemocratic elements in fulfillment of its subordination to the constitution. But in practice, FRG intelligence operated less nobly than its stated goals. Even though Basic Law provided broad protections for privacy and strictures against surveillance, the BfV and LfV sometimes flaunted these rules. By the same token, there was a consistent charge that domestic agencies like MAD and the BfV were "blind in the right eye," meaning that they regarded left-wing terrorism as a greater danger than right-wing groups. FRG intelligence agencies maintained a climate of anticommunism in which investigating left-wing groups received a higher priority than grassroots neonazis and other right-wing extremists.

Part of these problems for FRG intelligence can be traced back to the genesis of the BND: the Gehlen Group. Reinhard Gehlen was the chief analyst of the wartime Fremde Heere Ost (FHO/Foreign Armies East), a military intelligence organization devoted to analysis of the Soviet's military capabilities. Although not a trained intelligence agent, Gehlen made military intelligence more professional in ethos and the Abwehr's complicity in the 20 July Plot made Gehlen seek a closer relationship with the SS's SD. Gehlen realized that the writing was on the wall in 1945 and actively sought to carve out a niche for him in the postwar period. He ordered FHO archives to be carefully sealed in drums Bad Reichenhall in Upper Bavaria because he knew it was most likely that this region would come under American occupation. Gehlen also anticipated that FHO knowledge of the Red Army would be appreciated among his American captors and also drew up lists of potential postwar German partners for a future intelligence network.

Gehlen's bets were not wrong, as he struck up a positive relationship with his US Army interrogator Captain John Boker, Jr., whose recommendations that the US utilize FHO archives and German intelligence personnel filtered up the chain of command in the summer of 1945. American military intelligence on Soviet armies and capabilities was only rudimentary and both the archives and personnel were two prominent chips Gehlen could use for himself and his compatriots. Although the OSS initially rejected the utility of using German intelligence assets, the Pentagon did not, and under Gehlen's group, codenamed Bolero, started conducting intelligence gathering for the Army in 1946. The emerging Cold War soon allowed the Bolero group to mushroom into Operation Rusty, which would be a wide-ranging German-run intelligence network working for the US. By 1947, the Gehlen Group was operating out of the former Reichssiedlung-Bormann, which was an NSDAP retreat outside of Munich and it drew in an ever-increasing swath of personnel. This not only included intelligence personnel, but also key military and SS figures. As an October 1948 CIA report by James Critchfield on Rusty noted, the organization was not merely an auxiliary to the US by this time:

Another important consideration is that RUSTY has closest ties with ex-German General Staff officers throughout Germany. If, in the future, Germany is to play any role in a Western European alliance, this is an important factor. The issue than appears to be whether from a national viewpoint it is desirable for the U.S. [to] completely relinquish control of the resurgent Abwehr and nucleus of a future German General Staff which has been nurtured for three years by the U.S. Occupation Forces.

As with the Amt Blank maintained by elements of the military, the Gehlen Group maintained a shadow bureaucratic presence that would allow it to fully emerge as a separate organization rather quickly once the FRG had its sovereignty restored.

The semi-autonomy of the Gehlen group, something which their alleged American superiors often griped about, allowed for Gehlen to recruit members from a wide range of sources, including the SD, RSHA, and other NSDAP stalwarts. Critchfield noted that "no attention was paid to the character of the recruits, security, political leanings, or quality with the result that many of the agents were blown almost immediately." But the Gehlen organization had positioned itself as the de facto intelligence agency for the FRG, and as a result, it was able to shape the contours of both its personnel and operations, which at times worked against the spirit that underlay the structural division of the FRG intelligence system.

Although GDR's MfS has rightly earned a reputation as a successor to the Third Reich's network of police state terror and surveillance, it was actually the FRG's intelligence services that had the most former Nazis. Despite the reputation of the MfS as a "Red Gestapo", both the SED and Soviet military government paid a good deal of attention to the screening process. When the Soviet Military Occupation Government (SVAG) created K-5, a department of the Eastern zone's German police charged with monitoring internal politics and a precursor of the MfS, SVAG explicitly ruled out anyone with prior membership in either the RSHA or SD from membership in K-5. The MfS continued many of SVAG's restrictions and recruited among youth as a means to get people without a compromised past, even if it meant sacrificing professional experience over the short term. This type of caution was not really present in the Gehlen group, nor in its various successor organizations. The BfV and BND were allowed to employ "freelancers" as assets, and many of the veterans of the Third Reich turned to existing networks of wartime connections. Some of this freelance soon morphed into official membership in the intelligence agencies. Erich Wenger, a RSHA officer involved in counterintelligence as well as security in France, exemplified this career path. According to a 1963 CIA report on BfV personnel in light of an impending visit of BfV delegates to the US, the BfV contracted Wenger as an asset in 1950/51 under a pseudonym. Wenger soon became one of the BfV's chief officers for counterintelligence and when his SS background became known, he threatened to leave the agency for MAD, thus depriving the BfV of one of its most experienced counter-intelligence officers. The CIA memo noted that despite this problematic background, Wenger was a firm anticommunist, but Wenger's past soon caught up with him with a Spiegel expose about his involvement in anti-partisan atrocities in France, and his trip to the US was cancelled.

Despite the significant continuities in personnel between the Third Reich and the FRG intelligence system there really was not an attempt to recreate National Socialism inside these agencies. Participation in the FRG intelligence bureaucracy instead became a type of occupational Persilscheine for those men who were active in the former regime in which their wartime actions acquired an anticommunist gloss devoid of problematic politics. Gehlen's self-serving memoirs, translated by a then young David Irving, played up his wartime activities as an apolitical professional whom Hitler sneered at, but Adenauer did not. The presence of so many former Nazis inside German intelligence contributed to biases in investigation and a more casual disregard of existing laws, but such abuses were nowhere near as rampant as they had been under the Third Reich; nor were they as extensive as the MfS's methods of domestic surveillance, most of which were nominally illegal under the GDR constitution.