Tight shield to shield fighting is not the formation that made Rome.

The tight shield to shield style to which you refer could mean the Phalanx. The Romans used it very early. And there is no doubt that it was for its time, effective. The Romans reportedly adapted it from the Etruscans, but they also could have been exposed to the more advanced forms of the Macedonian style Phalanx in the Pyrrhric Wars. In any case, they used this formation until they met the Samnites (Samnite Wars 343-290), and the clumsiness and ineffectiveness of a Phalanx against a more mobile enemy (i.e. one not using a Phalanx) on rugged terrain became apparent. In short, they lost with it. And if Livy is to be believed, being adaptive, adopted the maniple as their unit.

The battle was exactly like one fought in a civil war; there was nothing in the Latin army different from the Roman except their courage. At first the Romans used the large round shield called the clipeus, afterwards, when the soldiers received pay, the smaller oblong shield called the scutum was adopted. The phalanx formation, similar to the Macedonian of the earlier days, was abandoned in favour of the distribution into companies (manipuli); the rear portion being broken up into smaller divisions. The foremost line consisted of the hastati, formed into fifteen companies, drawn up at a short distance from each other. These were called the light-armed companies, as whilst one-third carried a long spear (hasta) and short iron javelins, the remainder carried shields. This front line consisted of youths in the first bloom of manhood just old enough for service. Behind them were stationed an equal number of companies, called principes, made up of men in the full vigour of life, all carrying shields and furnished with superior weapons. This body of thirty companies were called the antepilani. Behind them were the standards under which were stationed fifteen companies, which were divided into three sections called vexillae, the first section in each was called the pilus, and they consisted of 180 men to every standard (vexillum). The first vexillum was followed by the triarii, veterans of proved courage; the second by the rorarii, or "skirmishers," younger men and less distinguished; the third by the accensi, who were least to be depended upon, and were therefore placed in the rearmost line. - Livy 8:8

Now that doesn’t mean the Maniple in itself is what made the Legion so feared. There are a host of other factors at play from Logistics, Camp structure, Training, Field Medicine, Equipment, and political will, and it goes on ad nauseum, but the Maniple is the go to formation when we commonly think of Roman Infantry. Polybius describes the Maniple as follows

Now, a Roman soldier in full armor also requires a space of three square feet. But as their method of fighting admits of individual motion for each man---because he defends his body with a shield, which he moves about to any point from which a blow is coming, and because he uses his sword both for cutting and stabbing---it is evident that each man must have a clear space, and an interval of at least three feet both on flank and rear if he is to do his duty with any effect. The result of this will be that each Roman soldier will face two of the front rank of a phalanx, so that he has to encounter and fight against ten spears, which one man cannot find time even to cut away, when once the two lines are engaged, nor force his way through easily---seeing that the Roman front ranks are not supported by the rear ranks, either by way of adding weight to their charge, or vigor to the use of their swords. Therefore, it may readily be understood that, as I said before, it is impossible to confront a charge of the phalanx, so long as it retains its proper formation and strength.

So a Phalanx head on, against a Maniple or anything else sounds like a winner, doesn’t it?

Why was the Maniple more effective than the Phalanx, which up until then had served other Empires such as the Macedonians so well? Our ancient source for this continues to be Polybius, The Histories, Book XVIII, Chapters 28-32:

Why is it then that the Romans conquer? And what is it that brings disaster on those who employ the phalanx? Why, just because war is full of uncertainties both as to time and place; whereas there is but one time and one kind of ground in which a phalanx can fully work. If, then, there were anything to compel the enemy to accommodate himself to the time and place of the phalanx, when about to fight a general engagement, it would be but natural to expect that those who employed the phalanx would always carry off the victory. But if the enemy finds it possible, and even easy, to avoid its attack, what becomes of its formidable character?

According to Polybius the Phalanx only works when the terrain is well suited for it. And when an enemy figures that out, they can outflank it, or turn its rear, and a flanked Phalanx, is fast a fatally flawed Phalanx

…The Roman order on the other hand is flexible: for every Roman, once armed and on the field, is equally well-equipped for every place, time, or appearance of the enemy. He is, moreover, quite ready and needs to make no change, whether he is required to fight in the main body, or in a detachment, or in a single maniple, or even by himself. Therefore, as the individual members of the Roman force are so much more serviceable, their plans are also much more often attended by success than those of others.

So Polybius tells us that it’s a matter of flexibility. If you have a big flat plain all conditions are perfect, and your enemy has no cavalry to speak of, or your cavalry can completely outmatch theirs and they stand still and take your charge, or charge back, then the Phalanx will roll over and cut up anything in it’s path.

But once flanked, it’s almost helpless. On rugged terrain it loses its effectiveness. In tight areas or wooded areas it becomes impossible to assemble. Without proper cavalry support it is easily outmanoeuvred by Maniples or by other cavalry.

Additionally, another tight shield formation you could be talking about is the famous Testudo. The Schildkröte, or terrapin. It’s great for keeping the arrows off, but you can’t do much else with it against a mobile enemy. It’s not an effective (i.e. mobile) active field formation. Cassius Dio outlines it’s ineffectiveness against Cataphracts at the Battle of Carrhae in Cassius Dio, Roman History, 40.22-23

For if they decided to lock shields for the purpose of avoiding the arrows by the closeness of their array, the pikemen were upon them with a rush, striking down some, and at least scattering the others; and if they extended their ranks to avoid this, they would be struck with the arrows.

When ever the Romans turtled up to avoid the archers, the Parthian Pikemen (Cataphracts) would attack to great effectiveness, or the Horse archers would charge. This demonstrates that the Testudo, while having its effective uses is not the field formation that made the Romans a feared opponent. It was handy in a siege, and a sure fire way to avoid a rain of arrows though.

Lastly, it has to be said that Maniple or no, the Romans could adapt and often triumphed by doing so. At Zama where they defeated Hannibal, they stretched their formation out in a line, in order to envelop the Carthaginians, a trick that had been used on them at Cannae. So in one of their pivotal victories, they didn’t need a Maniple to beat Hannibal.

Penultimately, Roman formations (Formations by Romans, not Maniples) were still in use long after Rome in the west was just a memory. The Byzantines, who were the eastern Romans, were still fighting very effectively well into the middle ages.

Why then after all of this, did the Western Europeans not simply revert to the Roman tactics that had proven so effective? Firstly, this part of the question is rooted in presentism. We see the Romans as the Hollywood badasses we want to see. The Western Europeans had been the Romans, and been the invaders, and times, weapons, and armies, had changed. As the middle ages came out of end of the classical period, they developed a chicken and egg system of either Heavy Cavalry that might have lead to Feudalism, or Feudalism, that with the addition of stirrups lead to heavy cavalry. (for the Chicken and Egg, please see, the Stirrup and its invention). And those cavalry did not lend themselves to any of the ancient methods of fighting. In any case, the mounted knight became the dominant figure for hundreds of years, until they gave way to the firearm.

Secondly, they didn’t have the knowledge of Rome that we had today. Surely they had some of it. But we have been “rediscovering” Rome for centuries. And even if they’d had it, in the west, the type of warfare they were waging was not the same of ancient world in terms of technology or opponents.

So in short, it wasn’t the Phalanx that made Rome, nor the Testudo. And the Maniple was the formation they used effectively until they met opponents that could beat that too, and they would either adapt, or eventually, be beaten, which in the end, they were. And the Western Europeans, who come after Rome, have their own methods that are shaped by their societies and their technologies.

This presents a small part of the marked difference in Phalanx formation and Maniple formation in an attempt to answer the question as presented.

Edited for spelling.

Because Roma was a highly centralized State that could afford a standing army while Middle age was mainly a decentralized era.

Fighting in formation requires a huge cohesion and a lot of training. This is therefore doable with a professional army with soldiers serving for 20 years with the same soldiers. Putting so much efforts into professionnal standing army also means you actually have a use for it all year because everytime this army is not fighting it means you are basically paying them to do nothing.

Medieval era was a lot more morcelled and the territory that constituted the former Roman Empire was ruled by hundreds of little lords, some ruling over two or three villages, others ruling over dozens but the fact remained that none of them had the money or the use for a standing army.

Armies were therefore levied strictly when needed among the local population, equipped according to the means of the lord and sent to fight, they would then be disbanded as soon as the conflict was over so the peasants can turn back to their main objective of growing food for their village. If it was serious enough the lord would hire some mercenaries to complement his own army. In any case there were no way an individual lord could get to the level of standardization or cohesion of the Roman Legions, especially since their peasants needed time to take care of whatever they grew, meaning training extensively together was very low in the list of their priorities. More importantly the individidual lord did not need that kind of army.

When a bigger invader threatened, the liege of the little lord would call his vassals to arm and they would all assemble their armies and go to battle together, each lord acting as an officer commanding his part of the army. As every part of the army was made of people who did not know each other at all, were equipped very differently according to the financial means of their lord and sometimes even spoke differently there was no way you would have made them fight as a highly cohesive group using advanced formations like the Romans.