Expanded from an earlier answer
Part I
The short answer is that they really did not see this as the brass in both OKH and OKW were the ones responsible for planning the invasion itself. Although Hitler was decisive in reorientating German military power towards the USSR, many German military thinkers were thinking along those same lines in parallel to the Fuhrer.
For the most part, a lot of German decision makers perceived the Molotov-Ribbentrop pact as a necessary evil, not as a new turn in German foreign policy. Both Göring and Rosenberg thought the Pact represented the baleful influence of von Ribbentrop whose anti-British biases made Hitler sign a deal with the Feind. The diary of Staff Colonel Eduard Wagner, OKH's Quartermaster-General, noted that the Pact strengthened the Soviet's strategic position at very little cost to them, and the Pact would only be useful if the Germans could prod the Soviets southwards into India and the Middle East, a move the Soviets were clearly not making. Other members of the German military were alarmed by the scale of Soviet occupation of the Baltics, which denied Germany its traditional strategic route into Russia. Soviet gains and Western intransigence by having the temerity to keep in the war after Poland's fall fostered an increasingly dour and pessimistic attitude about Germany's future in OKH circles. By October 1939, OKH was drawing up plans for an Ostwall of fortifications inside the General Government, one in which the SS claimed it would build using Jewish ghetto labor. Soviet insistence on Germany meeting its end of the economic pact also grew increasingly louder, isolating the few German proponents of a Rapallo-style rapprochement from mainstream opinion in these policy circles.
The Fall of France reversed this pessimism within OKH and the German decision-making establishment as a whole. In a June 1940 meeting with Lieutenant Colonel Reinhard Gehlen, who was tasked with constructing the Ostwall, OKH head Franz Halder noted that defensive measures should be minimal and "Whatever one has must be used for attack." Halder also tasked Artillery General Georg von Küchler in June to draw up contingency plans for a Soviet invasion. Halder's choice of von Küchler as well as von Küchler's chief of staff, Major-General Erich Marcks, is instructive. Both were quite senior Heer officers who had wartime experience on the Eastern Front during the First World War. In short, Halder entrusted the initial planning for an invasion of the Soviet Union not to the young turk officers promoted by National Socialism, but traditional officers that would make a sound operational plan. Küchler and Marcks's initial plan called for creating a German strike force in Poland that could be used in any contingency while larger preparations continued.
Thus by the time Hitler convened a meeting of senior military heads on 31 July at the Berghof to discuss a future invasion of the USSR, OKH had already formed solid plans in that direction. Although Hitler had likely conceived of a Soviet invasion independently of OKH, the Berghof meeting unchained the German military to expand its preexisting plans. The disastrous performance of the Red Army in Finland and captured Polish intelligence on the Soviet military led OKH to fatally underestimate the military capabilities of the Soviets. For an army whose operational culture that stressed boldness and offensive encirclement battles, the fact that the Soviets had deployed large parts of their military in their newly-won gains ahead of their lines of supply and lacked fortifications made the Soviets a very tempting target for the Germans in 1941. By August 1940, Marcks had outlined the draft of his plans and they were quite ambitious:
After we take Kharkiv, Moscow and Leningrad, cohesive Russian armed forces will no longer exist. Complete occupation of this territory is not possible and not necessary. Rapid troops and inf. div. advancing by train will be chiefly responsible for maintaining the pursuit. Time required: 2–4 weeks. Total time required for the campaign until the eventual objective will be between 9 and 17 weeks. If the Soviet regime does not collapse or sue for peace, it could become necessary to continue advancing up to the Urals. Even if Russia is not capable of actively waging war after the destruction of its armed forces and the loss of its most valuable European regions, it could still, supported by Asia, remain in a state of war for an unforeseeable length of time.
For its part, OKW drew up a set of parallel plans for a Soviet invasion, but after the 31 July meeting and by Lieutenant Colonel Bernhard von Loßberg. The result was Operational Directive No. 21 which meshed somewhat with the plans developed under Küchler and Marcks's aegis, but OKH had misgivings about the main thrust of the attack and the overall strategic orientation of the German advance.
While this disagreement played a part in Barbarossa's failure, there was a general consensus by the summer of 1940 within German military circles that a war on the Soviet Union was necessary either to knock Britain out of the war by depriving her of her last hope of challenging German hegemony (the USSR) or to secure Germany from the Bolshevik threat. The dissension that existed within the German military was mostly about the direction of the attack on the USSR, not the actual attack itself. The polycentric administration of the Third Reich sharpened this dissension as multiple power blocs worked to achieve their own ends and found that "working towards the Fuhrer" was a means to achieve victory over their rivals. Because of this murkiness, Halder's selectively edited diaries and postwar memoirs allowed the onus for the genesis of the Barbarossa to be placed directly upon Hitler's shoulders and 31 July meeting, but in reality OKH had been thinking along those lines for quite some time without direct prodding by Hitler.
The only real power bloc inside OKW that voiced any real apprehension about Barbarossa was the Kriegsmarine. Much of the Naval Staff's planning was directed towards defeating Britain and the various war material the Soviets provided severely weakened the British blockade of the continent. Kriegsmarine head Erich Raeder expressed "serious doubts" about Operational Directive No. 21 given that Germany was undertaking a massive military operation before the British had been defeated. But Raeder's doubts about Barbarossa need to seen within the context of the polycratic Reich. The navy had been pushing for its own grand reorientation of the German war effort towards the Mediterranean and the Atlantic and looked with alarm the increasing likelihood that America would join the war. The Naval Staff felt that by destroying Britain's Middle East position, they would be able to knock Britain out of the war and further isolate the US. Naval discouragement of Barbarossa does not equate to doubts about the whole enterprise or its justifications. The Naval Staff's chief of operations Rear-Admiral Kurt Fricke produced a memorandum "Observations on Russia" in 28 July 1940 that Soviet expansion in the Baltic was an existential threat to the Reich:
The safety of our homeland, however, also demands in the future the development of spatial unassailability, i.e. an expansion that would prevent any unchecked surprise penetration of vital parts of German soil, a deep forefield . . . which would at the same time provide the necessary living-space for the German nation in the future. . . It moreover demands the greatest possible economic autarky, especially in commodities vital in wartime (e.g. oil, foodstuffs). The further development of Germany requires raw materials and also, to the same extent, markets for her manufactures. For both these purposes Russia is ideally suited . . . The Baltic must become Germany's "Mare nostrum".
Fricke's conflating economic security with national socialist paradigms like Lebensraum was emblematic of much military thinking throughout OKW when considering the USSR. Küchler would claim that the ideological and racial "abyss" that separated the USSR and Germany meant it was necessary to invade to bring "about a long-term pacification of Europe, in which the German people controls a space that will ensure its food supplies and those of the other European states." For many of these German planners, war with the Soviet Union was both strategically necessary and just according to the racialist precepts espoused by National Socialism. Chief of Luftflotte 4, Colonel-General Wolfram von Richthofen's wrote in his diary in January 1943 that he was rereading Mein Kampf's chapters on Lebensraum and that they were:
Still very interesting and, even in today's situation, providing the answers to nearly all questions. Shall see to it that these observations are given greater prominence among the forces in the whole area.