The purpose of the Square formation is for cavalry defense. This picture depicts a small company sized infantry square, where you'll see officers and any supply wagons inside the square while the men are in two rows, with the front row kneeling with bayonets facing up while the second row will be ready to fire. The idea is to protect the infantry unit from all sides and not have problems of horses charging in. Now, you might say "the horses can just charge in, it's hollow" but that's kinda the point, the horse isn't a dumb animal, and it'll stop rather than charging into a fence of steel, either the rider will fall off or have a hard time getting composure of the horse.

However, you do give one important thing, it is very easy to kill several people at once. Infantry squares are a very easy target for artillery and infantry in line formation. As such, this is only a defensive formation should only be used when necessary.