Why did Germany invest so much in a U-boat fleet as opposed to a surface fleet? Was it because the British fleet was just too large? Why did they focus on U-boats while it seems like every other navy had small submarine fleets. Was there a technological edge the Germans had in their U-boats that other nations could not replicate in their submarine fleet?
Are there any good sources on German U-boat theory/ tactics?
Right so. The shortest and simplest background is that while Britain's navy had made worldwide Empire possible, the Germans in the run-up to World War I were trying to rapidly catch up to the Brits. Other powers participated, and a full-on international arms race was in effect. The heavyweight boxer of the modern international navy in the run-up to World War I is the Dreadnought-style, all-big-gun battleship.
The Germans built a top-tier fleet for themselves, but were never a true blue-water force - British doctrine saw to it that they consistently out-built them. Their main fleet of battleships never really got in gear throughout the war, but Germany had a world-wide naval presence with other types of ships. But make no mistake, the High Seas Fleet was mighty, and was hella imposing.
Now, submarines are excellent for two things: sinking large capital ships despite a massive disparity in cost and resources needed.
The second is commerce raiding. Both sides in the first world war attempt to blockade the other: Britain is an island and relies on imports heavily, and Germany has food and resource needs of their own. Submarines are excellent hunters of merchant vessels. Accompanying them on their mission throughout the war are various smaller vessels, the Kaiser's cruisers, and various battlecruisers/pocket battleships.
Germany realizes it cannot compete on the high seas over the long term - they lack the ability to get its fleet into open ocean without massive confrontation. The submarines, however, result in "more than 12 million tons of shipping - around 5,000 ships."
So, they don't quite starve out the Brits, but they do a shit ton of damage for comparably little cost.
Between the wars, Germany has its navy crushed by Versailles, and basically building what they can until Hitler comes in and rearms with the new Kriegsmarine. The Germans attempt to implent Plan Z, an attempt to compete with the British by building 10 battleships (think Tirpitz and Bismarck, the only true battleships the Germans got out there in the second world war, compared to the much more equal footing of the Kaiserliche Marine's fleet) and four aircraft carriers, to be completed by 1948.
Obviously, war breaks out in 1939, and the Germans pretty much scrap this plan. You end up with some ships built that were ordered before the Plan was even adopted, and it basically goes straight to the backburner. Take a look at the Prinz Eugen, though. Beaut of a cruiser that was tough as nails. Barely sank after getting an atom bomb dropped on her.
Anyway, Admiral Doenitz (excuse me, it's late, I'm stoned, and I can't find my umlauts) reckons that it would take him 300 U-boats to strangle the British into starvation and defeat. Let the commerce raiding begin again.
Effective anti-submarine warfare allows the British to defeat the U-boats soundly in this conflict, and while Doenitz gets his 300 boats, it's late 1943, four years too late, and his side has failed in its mission. While the Germans sink 3,000 ships in the war and put a very real scare into the Allies (hell, they put plenty of 'em right up to the US' doorstep, feasting at the mouth of the Mississippi for awhile).
Anyway, U-boats were cost-effective and effective-effective in Germany enacting its war plan. Hell, they were pretty much the brightest spot of the war at sea for the Germans. The surface fleet were penned in far more than they weren't, and the U-boats were a threat all over the place, until the end of the war they could be feared.
So, while the British had submarines, they basically didn't need them to accomplish their goals at sea - the German surface fleet was penned in and the blockade left Germany short of food and resources in both wars. Look to the Pacific War in World War II for a lot more submarine action with modern boats, but I know almost nothing about that part of the war.