I am not sure if this is how history is portrayed or real history in play. Whether I encounter a story because older battles like WW1 or WW2, they are often depicted are incredibly large with hundreds or maybe thousands present and incredibly destructive like the trench of Western Europe in WW1, or the all in strategy of WW2 like aerial bombings or D-Day.
Then when depicting battles from the 50s onwards (unless you count the Vietnam war with Napalm bombings and underground tunnels), this is where we see a shift in how battles are played like espionage missions or small skirmishes in the Middle East.
How did this shift in military conflicts change as time progressed despite the advancement in military strategy and technology?
The shift you are observing is because of the advancement of military strategy and technology, not despite.
More to the point, you are seeing depictions, not the actual events.
Historically, battles were fought in what we would call linear tactics. When using wielded weapons or muzzleloading weapons, the strength of those units were directly related to the amount of how well those weapons could be massed at the point of contact. A person with a sword is useless when it comes to breaking a shieldwall 100 feet away. Nor can an individual have a lot of impact when they can only really engage one opponent at a time. You have to have mass.
That meant you were in very close contact with the rest of your force. They were next to you in pretty much every direction. And as there was no mass communications, unit size was pretty limited to what could reasonably be controlled by shouted commands.
Now, WWI and the early 20th Century was when this really started to change. The use of rifles with magazines and the ability to engage a number of opponents at a far greater range lead to an overall thinning of lines and spread out of soldiers, but they still needed to be within the range of sound commands as they did not yet have tactical radios at the troop level.
As the 20th Century went on, the introduction of tactical radios detached the need for soldiers to be within range of sound commands and the introduction of easily carried automatic weapons meant smaller and smaller units had far more massive manpower than units had previously.
The battles were just as large in a few cases. The Tet Offensive had well over 1,000,000 combatants. It was just that the battle spread out over hundreds of miles instead of being compact. You rarely see anything more than what is within visual range. When you get to the Persian Gulf War, you find that massed units were just magnets for artillery and air attacks. In fact, I would go so far as to say that a general prays for their enemy to mass all their troops in a small confined area as that is a gift-wrapped present that rarely happens. Those units simply would not last long in any combat situation.
Modern battlefields are mostly notable for how empty they feel. It is much harder to attack when you can't see the defender. It is much harder to defend if the enemy can call in a guided missile on your defensive position if they spot you. It is much harder to stop an attacking force if you can only take out its soldiers 4 at a time (roughly the size of 4 man fire team). Same applies to taking out a defense where it can be a single defender anywhere calling in artillery.
Short answer to your underlying question: Rapid mass communications to the troop level combined with massive amounts of firepower available to an individual soldier makes mass concentrations suicidal and unnecessary.