First: castrum is not typically used in Latin, except in very rare cases, and those probably confusion with the similar castellum, which does appear in the singular. The Latin noun is always plural, castra, though it refers to a single camp.

For the average run-of-the mill castra, there were no "buildings." The structures inside were tents, and the whole thing was put up and taken down in a single night on the march. The more permanent forts, like those along the limes in Imperial times, were constructed on analogy with the marching forts, and more or less took the same shape.

There are practical considerations for both the temporary and the more permanent forts. The spaces between the tents/buildings afforded rapid internal movement from any one side to any other side. Men could move quickly down the "lanes" (the Romans called them viae, 'roads'). Internal mobility was an important feature of successful repulsion of attack, since reinforcements could be moved rapidly from point to point. The lanes also afforded quick mustering and departure from any of the four gates.

You find it hard to believe that the castra could be built within a day, but I think you have the wrong idea about what it looked like. See here, though this pic probably gives TOO much space inside the walls. It was not only built in a day, but often within a couple of hours, or even less. Every soldier knew his job, where to dig, where to pile up dirt, and much like ants, and well-trained legion could erect a structure like a castra with terrifying speed. And contrary to what you are suggesting, the space between the rows of tents makes construction and organization go faster, because all the tents can be erected at roughly the same time, instead of each row in succession in the case of them "touching" one another. I also don't understand why a Roman army would need to "conserve space" on the march. Space from what? When the Romans are building a castra, they typically own the countryside far and wide.