If you look at the map of Europe in 1939, you'll see that Germany and the USSR didn't share any border. So, if Hitler invaded the USSR, he'd need to do it through some other country. Theoretically, he could've staged troops in the exclave of East Prussia and invaded through Lithuania, but that would've extended supply lines a lot (since East Prussia didn't have enough resources to support the army by itself.)

So, even if Hitler would have been satisfied with invading the USSR at once, he would have needed to either invade Poland first or recruit them as an ally - and since they (quite sensibly) weren't interested in alliance on any terms Hitler would have entertained, invasion or changing goals were his only options.