Copying my older answer to the same question:

You're making a number of assumptions that don't really apply to any historical combat situation. Here are some important things to consider.

 

1. A formation is not just a bunch of dudes

For your suggested solution to work, every man in the formation must be equally capable of fighting in the front rank, on the side, or in the rear; they must be ready to change their position and role at need. Some modern computer games render the wheeling of a formation by simply tracing every warrior's shortest path to a new position that would create a change of frontage for the unit. Historically, this is not how formation evolutions work. Even on today's military parade grounds, this is not how formation evolutions work.

Battle formations that have been taught unit drill to any extent are in fact carefully regimented structures of men, with officers in predetermined positions, and often with selected men fighting in the front and rear ranks. If the men in these formations were to simply turn in place, their entire battle order would be irreparably ruined. Officers would no longer be in position to lead or give commands; the strongest and bravest men would be sidelined; the least capable troops would suddenly be required to face the enemy. Instead, from at least the days of Classical Sparta onwards, commanders devised sometimes complicated manoeuvres to ensure that those who were placed in front would remain in front, and those who were placed at the rear would remain in the rear. This was the only way to ensure that the formation would remain intact and in good order. Individual turning would result in chaos. If an enemy were to appear in the flank or rear, the formation would have to be wheeled or countermarched as a whole.

2. A formation needs training and space and time to be able to wheel or countermarch

Since it is essential for a proper battle formation to retain its original order to remain effective, the evolutions necessary to retain formation while wheeling or countermarching must be carefully drilled. While Spartan hoplites and Macedonian pikemen were indeed prepared for this, many armies in history have consisted of levies sent into battle with little training, and even the better trained ones can have patchy records of carrying out their drills in the heat of battle. Even on a parade ground, wheeling and countermarching takes time and careful attention; even the best drilled troops will lose some of their cohesion and will need to dress the lines after their evolution has been completed. For untrained militia like the average Greek hoplite army, such manoeuvres are pretty much unthinkable. For any army, carrying them out in confined spaces with nearby enemies is extremely difficult. The only option that remains is indeed for individuals to turn in place - which means losing sight of their commanders and suddenly finding themselves in the front rank with the enemy bearing down on them.

3. The enemy isn't going to just stand there and let a formation change its facing

As noted, formation evolutions are complex and require time and attention. If the enemy suddenly appears in the flank or rear, it is certainly possible for a formation to rearrange itself to face them - but the enemy isn't going to let that happen if they can do anything about it. Either by threatening them with their presence, or by charging in to pin them, the enemy will try to spoil any attempt to react to the danger.

If a unit is already engaged in front, there is nothing it can do to change its facing; any formation evolution would expose their flank or rear to the enemy already in front of them. A unit that is already in combat is inevitably going to focus on the problem at hand. It can do nothing else until that problem goes away. But even a unit that isn't already engaged is going to have to be wary of other nearby enemy formations. If the battle line is advancing upon a visible enemy battle line, it is irresponsible to expose the flank of one unit by turning that unit to face a threat from the flank. Either way, the unit leaves itself open to flank attacks; unless it is supremely disciplined, it will not be able to face two ways at once. This is why, when an enemy unit appears in the flank or rear, an infantry formation is likely to either stay the course regardless, or panic.

4. Fighting the enemy is not necessarily the priority of the troops

It is easy for us to chide warriors on historical battlefields and tell them what they ought to have done. When the enemy is in front, it's as easy as advancing to fight them; when an enemy appears in their flank, why don't they just turn and fight them? But human warriors are not mere lines of computer programming. They are not hardwired to keep fighting until either they or the enemy are destroyed. Instead, when they find themselves in a situation that they know they cannot resolve - such as the problem of having to fight enemies in two directions at once, without having a chance to adjust the formation appropriately - they are very likely to decide that surviving is far more important than fighting the enemy.

The individual's agency is most apparent here. If you are a spearman on the flank of a formation that is advancing into battle, your entire being is focused on the impending fight to your front. You may already be able to see or hear your friends at the front, screaming and stabbing, killing and dying. You are not going to be able to handle much more than the stress and terror of having to face the enemy directly in front of you. But then suddenly holy shit they're in the flank as well. What do you do? Do you turn 90 degrees, and go "oh well, it's stabbin' time"? Knowing that you are turning away from your buddies, who are continuing their forward advance and will start moving away from you? Once you face the enemy, you have to take it on faith that your friends have your back - but last you saw, they were moving in another direction. And the enemy is coming closer and they're angry and howling and they just cut off one of your paths of retreat.

It's at this point that many a spearman will decide they'd rather leave the battle to the reckless and the foolish. Much as formation combat isn't about mere individuals, battle isn't about mere combat; much more than that, it is about a willingness to fight. Discipline and training can provide a substitute to some extent, but even so, humans instinctively try to survive. For this reason, battles generally aren't won by wearing down the enemy until there are none left (like in games or movies), but by breaking the enemy's will to fight. Appearing suddenly in their flank or rear is an attack on their minds as much as on their formations, and few are tough enough to withstand it. The Classical Greeks were well aware that even small groups of men could have a devastating impact on the enemy if they could just make sure they appeared at the right time:

A fresh attacker always brings more terror to the enemy than the one he is already fighting.

-- Thucydides 5.9.8

 

This is the principle of outflanking and the reason why outflanking works. You are thinking about this too much as a matter of a single, infinitely dedicated warrior's personal environment. In reality (and games often struggle to reflect this), battle is a social activity; it is something that groups of humans do; groups of humans crave order and cling to the familiar and expected and they hate and fear nothing more than to discover they're in more danger than they thought.

Not a historian: so I had written this long ass post as a reply to another comment that got deleted as I was typing. And I don't want my time to go to waste so I'm going to post it anyway since it actually is somewhat relevant to your question:

Hannibal did at Cannae something absolutely no one at the time thought possible: the double envelopment of a numerically superior force by an inferior one.

So Hannibal positioned his troops in a convex formation with his weakest troops in the centre. This was on purpose. Normally the Roman army arrayed itself in a checkerboard pattern to allow more mobility but seeing the weak Carthaginian centre, the commanding consul, Varro, took the bait and forced his troops into a denser formation to break through that centre. Hannibal, however had his strongest troops on the flanks and he positioned himself in the centre to say to those troops that he had confidence in them and if they broke, he would die too. His presence prevented exactly what the Romans were hoping to achieve. So when battle commenced, the centre was pushed back step by step, which encouraged the Romans to pack in tighter, and all the while flanks held their line and did not move. By allowing the centre to buckle slowly and without breaking, Hannibal allowed the convex to eventually become a concave with the Romans now facing enemies on three sides.

This is where the Numidian cavalry comes in because they engaged the Roman cavalry at the start of the battle and after chasing the Roman cavalry from the field, they wheeled around and attacked the Roman infantry from behind, essentially boxing them in from all sides now.

This is reason Hannibal is considered one of the greatest tacticians of his time and why Cannae is one of the most studied battles in military history. He did the impossible. (Scipio Africanus repeated the feat against Carthiginians in Iberia a few years later)

The slaughter that followed was horrific with the Romans packed so tightly they could barely lift their arms to swing a sword. (My own personal opinion is that there was possibly a crush in the centre of that mass of soldiers, like we see in "stampedes", so many probably died by asphyxiation) One of the reasons Hannibal allowed the slaughter to continue as long as he did was because, being a numerically inferior force, he did not have the manpower to care for so many POWs and he couldn't just let them go back to Rome only to rearm and take the field again. So they died.

This I think also answers OP's question a little on the dangers of envelopment because the moment the Romans were enveloped, they tactically and psychologically lost the battle. All that was left was the slaughter.