I think one thing which many downplay was just how static the soldier's experience of WWII could be. It was not all, or even mostly, aggressive attack and counterattack like popular culture implies. German offensive successes in 1940-42 (what most people call Blitzkrieg, although the Germans never called it that) and the Allied breakout from Normandy have both given the impression that such sweeping moves were routine, but attacks were almost always slow and grinding. In fact combat in WWII in my opinion bears much more resemblance to the stereotypical view of WWI than the view espoused by films and games set in WWII. It's inarguable that the belligerents of WWII went into the war with methods that had been developed in the latter stages of WWI--in particular WWII re-emphasized the necessity of tremendous amounts of supporting artillery fire to neutralise enemy defences and 'shoot' the attacking troops into enemy defences. What's more, the pattern of fighting became more slow and grinding as the war went on--the Germans abandoned anything resembling Blitzkrieg and began to build their tanks as mobile bunkers and emphasize flexible defence by groups of heavily-armed infantry; the Allies relied more and more upon the brute fire-power that their industrial superiority could supply. British troops eventually became so reliant on this support that the routine reaction to coming under fire was to go to ground and wait for fire support. The same kind of tactics used in Normandy, with any attack supported by tons of shelling and any enemy counter-attack smashed by artillery as soon as it began, resemble closely those developed on the Somme, where artillery was used in just this manner--the effect on the landscape was similarly destructive. In WWII soldiers fought in basically the same way their fathers had fought in 1918, with the addition of more mobile tanks (of which there were always relatively few compared to infantry), portable radios, reliable motor vehicles (though the only fully mechanised army at the beginning of the war was the British Army), and larger firepower.

This continuity is summed up really well by John Ellis in The Sharp End of War, which is a great book on the soldier's experience which I'd recommend you read if you can.

Above all, perhaps, the two world war have in common the shovel and the entrenching tool. From 1914 onwards the paramount fact in war was firepower of such intensity that only in holes in the ground could the front-line soldier even begin to feel relatively secure. Armoured vehicles briefly robbed defensive tactics of their supremacy, but by 1941 at the latest anti-tank gun, the mine, and a little later the bazooka/PIAT/Panzerfaust, had done much to restore the balance. Attacking remained a hazardous and slow procedure, only to be undertaken when necessary and with the maximum amount of fire support...though World War II never had trench systems as static or elaborate as those on the Western Front, the individual soldier nevertheless spent much of his time burrowing into the ground...if defensive firepower was no longer sufficiently predominant as to make only the most trivial gains possible, it was still quite enough to make progress agonisingly slow and to necessitate constant retrenchment after each desperate bound. Even when troops were not themselves on the defensive, actual advances across the battlefield were sporadic and slow, each attack a frenetic spasm in the troglodyte routine. In the front line at least 90 per cent of the infantryman's time was spent under cover either on the defensive, often under bombardment, awaiting an enemy attack, or nerving himself to the prospect of going forward.

I'd also recommend Time to Kill: The Soldier's Experience of War in the West, 1939-1945 for excellent, more academic assessments of the key belligerents. Otherwise there are countless works which look into the minutiae of how individual armies operated and how the precise forms of battle differed greatly, as others have pointed out.

there is a huge emphasis on the significance of infantry combat with troops engaging in combat only occasionally receiving support from armor, air, or artillery

Artillery is in a completely different category from armor and air here and rare would be the battle in which neither side drew on artillery support. WW2 armies were equipped with mortars at the company or even platoon level, so even if a unit were completely isolated they'd have something. And in normal circumstances, "real" artillery pieces would be firing on both sides. Particularly in the attack, going in without artillery support against a prepared position is just suicidal.

While I'm sure that the absence of armor and air support is more a function of technical limitations or design choices in the video games you're thinking of, it is true that the importance of armor and air on the WW2 battle is easy to exaggerate. Keegan's Face of Battle has some good remarks on this in the concluding sections. Most of the fighting men of most "armored" divisions, especially in the later war, were actually foot-slogging infantrymen, and "tank" battles were "heavily intermingled with confused infantry combats of a kind little different from those with which soldiers of the First World War had experienced in many of the great offensives."