There were several factors that appeared to give bombers the upper hand in the inter-war years, particularly when Baldwin delivered that speech in 1932. In May 1917, 21 Gotha bombers bombed Folkestone, and though 74 fighters were launched to intercept them, not a single one made contact. There were efforts to build sound-based early warning systems using [acoustic mirrors] (http://upload.wikimedia.org/wikipedia/commons/9/94/WW1AcousticMirrorKilnsea%28PaulGlazzard%29Jan2007.jpg), but these weren't very effective, especially as aircraft got faster. Some theoretical work had been done on radar, or Radio Direction Finding (RDF), but the first actual demonstrations in the US, UK and Germany were in 1934 and 1935, Britain started to put the Chain Home network in place in late 1936. Without radar, and just as importantly a command and control network to rapidly collect and disseminate information, just finding enemy aircraft would have been a major challenge.

If you could find the enemy bombers you then had to catch them, and fighters and bombers leapfrogged each other in performance between the wars. The Hawker Hart light bomber of 1930 was faster than the Bristol Bulldog fighter; the Bristol Type 142 monoplane with retractable undercarriage, developed into the Blenheim bomber, first flew in 1935, outpacing the biplane fighters of the RAF. Germans Schnellbombers (literally "fast bombers") such as the Dornier Do 17 and Junkers 88 were developed along the same lines, to outrun potential interceptors. If you could find and catch the bomber, you still had to shoot it down; instead of avoiding the enemy another possibility was that bombers would be able to mount a heavy enough defensive armament to ward off enemy fighters, especially when the front line fighters of 1932 were armed with only a pair of light machine guns.

Baldwin wasn't entirely incorrect; though, as you say, daylight raids suffered unsustainable losses, when the Luftwaffe and Bomber Command switched to night attacks it took several months to introduce effective night defences during which the bombers did almost always get through, though not always accurately (particularly in the case of Bomber Command), but the more apocalyptic predictions of devastation such as Baldwin's speech, fuelled by fiction from the likes of H G Wells, generally proved beyond the capabilities of early war bombers. The Schnellbomber concept was also embodied successfully in aircraft such as the De Havilland Mosquito, and the few examples of the Arado Ar 234 jet bomber that reached service.

Interwar thinking was largely influenced by Giulio Douhet, an Italian General who was a radical proponent of strategic air power.

In 1921 he wrote Command of the Air, in which he proposed that a fleet of strategic bombers would render ground-based warfare obsolete. The bombers would simply overfly the front and strike at the enemy's vital centres - industry, communications and infrastructure, government and of course civilian populations.

The basis for this thought was that at the time, there was no radar so no effective early warning system by which a defender would be able to mount an effective interception of the enemy bomber force, even if their interceptor fighters were fast enough to catch the bombers.

And the latter point is really quite important because until the introduction of the new generation of monoplane fighters (Hurricane, Bf109 etc.) there was little if any speed advantage to be had between fighters and bombers as shown by a few examples below:

Fighters:

Gloster Gladiator - 1935 - 210mph

Hawker Hind - 1935 - 185mph

Focke-Wulf Fw56 - 1935 - 158mph

Arado Ar76 - 1936 - 138mph

Bombers:

Junkers Ju52 - 1931 - 195mph

Junkers Ju86 - 1936 - 260 mph

Vickers Wellsley - 1937 - 180mph

Bristol Blenheim - 1937 - 198mph

Dornier Do17 - 1937 - 255mph