The short answer is, Europe by and large didn’t have any significant oil fields. Just take a look at production figures for 1938 and you’ll see the hole the Germans were operating from.
All of non-Soviet Europe in 1938 produced 7.997 million metric tons of crude oil. 6.6 million tons of this came from Romania, most notably from around Ploesti. This accounted for a bit under 3% of global crude production.
In contrast the Soviets produced 28,895 million tons. While the US dominated global production with 164,302 million tons. The other major players included Venezuela (28,071), Iran (10,359), the Dutch East Indies (7,398), and Mexico (5,877). There were another 7 countries and territories that produced 1,000 million tons or more in 1938. All of those were either in the Americas or South Asia comfortably out of German reach.
So it’s easy to tell why the Germans and Japanese both were so concerned about oil. Both could (and did) secure about 3% of the 1938 production areas, while the Allies controlled access to the other 94%.
All figures taken from the Statistical Year-book of the League of Nations 1936-1944
The answer lies in the question itself. It is true, that countries controlled by Germany in the years 1939-1945 had some oil, but to wage a successful war against several strong enemies using advanced technological equipment (both for combat and transport), 'some' doesn't cut it. One In such situation one need a whole lot of oil.
The problem with European countries is that they really have very little natural oil resources available and even though Germany and Italy were in early 1941 either controlling or allied with almost all European countries save for Sweden and Great Britain, very few of the territories they could count on had any significant amount of oil. Actual production notwithstanding, if you check the modern EIA/OPEC estimates of the oil reserves around the world, you will notice that the only country with significant oil reserves in Europe are Russia, Azerbaijan, Norway, United Kingdom, Romania, Denmark, Italy, Belarus and Ukraine. All other countries, Germany included, have very small reserves, much of which were still unknown or economically unfeasible to use in 1940s resulting in poorly developed oil extraction industry. But this is a present state. In 1939 (or 1941), what is now Azerbaijan, Ukraine and Belarus, were constituent parts of the USSR, along with Russia; Norway and Denmark began their exploitation of sea oil reserves only in 1980s; Italy followed suit in early 1990s and United Kingdom (controlling large part of the Middle East at the time) was, for obvious reasons, well beyond Axis' reach. This leaves us with USSR and Romania. And here is where the things becomes interesting.
After the signing of the German-Soviet Trade Agreement on 19th August 1939, and the German-Soviet Commercial Agreement of 11th February 1940, USSR became the larges supplier of oil to Germany. It is estimated that between signing the agreement and severing the ties due to the breakout of hostilities on 22nd June 1941, USSR supplied Germany with over 900.000 tons of crude oil and oil products that formed 67% of the German stockpiles at the beginning of the Fall Barbarossa (invasion of USSR). How small these stockpiles were is evident when we take into account the fact, that in October 1941, after four months of operations on the Eastern and African fronts, the stockpiles fell by 33%. This is why capturing the oil fields in either Caucasus or Middle East was crucial to the war effort.
Also, since the early 1940 Allies were planning the aerial campaign aimed at disruption of German supply in oil and petroleum products (fuels, lubricants, synthetic rubber etc.) by tactical bombing of the petrochemical industry targets. First attacks started in May 1940 and continued throughout the war. The main targets were oil rafineries, depots and chemical factories in Germany. Later, the attacks were also directed to Romanian targets, such as oil fields and refineries in Ploiești and Câmpina or port in Constanța, further crippling the oil production available to Germans.