It seems like the major downfall of the phalanx was its inability to protect its flanks and rear from attack. Why did they never think to turn the men on the flanks 90 degrees and the men in the rear 180 to form a square of pikes? Movement may have been further restricted, but then walking backwards or sideways slowly doesn't seem like that much of a feat for trained soldiers. Maybe I am missing some key point but having islands of heavily armored men with 18 foot pikes sticking out on all sides seems like a much better system once the weaknesses of the phalanx became clear when fighting the roman manipules and legions.
They actually could rather quickly shift aiming their pikes to the rear or flanks if they had a clear view of the entire battle.
The problem is that the soldier on the ground does not have an aerial view of the battlefield, especially when packed in the middle of the formation. So those in the middle might not be aware that the flanks or rear are already being attacked and orient their pikes in time.
Spreading the pikes in all direction moreover is a diffusion of effort - if you have 100 pikemen for instance you will have 25 pikes facing each face of the compass. By comparison an enemy that puts all his pike forward can concentrate 100 pikes to your square's mere 25. Density of pikes in fact matters quite a bit as opposed to merely the direction they are facing - especially when Phalanxes tended to rely on literally pushing the other Phalanx into retreat. Having 100 guys doing the pushing versus 25 is an easy result to predict..
Finally, such a "square" formation would be practically immobile. Pikemen on the flanks will have to walk sideways with a very long two-handed weapon, which will very much be difficult at best and comedic at worst. This is why Napoleon-era squares were also immobile.
An infantry square like you're describing is a defensive formation, plain and simple. Is it effective? Yes, your soldiers can't be outflanked. The downside? You can't do much else with it. Your formation becomes a static element on the battlefield and the enemy can attack you with arrows, a cavalry charge, or both. During the Battle of Carrhae, the Parthians wore down Crassus' hollow square with both of these. Your men can only endure something like that for so long until fatigue and attrition sink in and gaps in your formation appear.
It also still doesn't solve the other major vulnerability with the phalanx: maneuverability. Walking laterally or backwards while maintaining a tight formation doesn't sound too difficult until you try getting hundreds of men in heavy armor, carrying a long spear, over uneven terrain to do it. The Battles of Cynoscephalae and Pydna demostrated that trying to reposition or maintain a phalanx over uneven terrain proved too difficult to do in the face of more maneuverable formations like the Roman maniple.
I'd just like to make note of one thing; many people criticize the phalanx as being overly vulnerable to flanking. This is true. However, flanking has been an effective way to defeat an army since armies were formed; it may be difficult for a phalanx to turn and face a flanking force, but if you have already been flanked, then it probably won't matter whether you have pikemen or legionaries.