At the most basic, any building structure needs to be stable enough to hold it's own weight and withstand the forces of wind, rain and snow to be viable as a building. The way this is achieved varies, of course, depending on the available material with which to build, the tools which are available to use and the climate of the region in which the structure us built.

To achieve a sound structure it's greatly advantagous to build in a regular shape, not only to distribute these weights evenly, but also because you can produce or gather similarly shaped building materials to use in your building. There are early examples of more irregularly shaped structures, like the famous Skara Brae settlemet in Orkney, where the inhabitants used differently sized stones to build their dwellings, but even there you can see the connection between availability of material and form of the building.

But when using wood or stone (or brickwork) as a building material - again, depending on the region - you can choose and form your materials into regular pieces to achieve a regularly shaped building. The most basic regular shapes are circles, squares and rectangles. Both circular and rectangularly shaped buildings are common across many different cultures, and to answer your question on why one form was used is linked to the aformentioned available materials and the function of the building itself. Both shapes have their unique features for inhabitation: circular buildings like yurts can be constructed around a single structural pole, which makes them easy to dissassemble and transport. On the other hand, a circular building can not be easily enlarged, as you would need to basically rebuilt the whole thing to add more square footage (by enlarging the circle).

Rectangular buildings, on the other hand, can be easily enlarged, by adding length to one side of the building, or grouped closely together by adding another new building or room to an already existing wall. Rectangular buildings are also easy to construct from wood or stonework.

This is getting very technical, but I wanted to emphasize that the form of the buildings we are familiar with is ultimately the result of the regional conditions in which they are conceived. There are often parallels on the shape of buildings across very diverse cultures that stem from similar conditions, like the shape of the roofs of traditional houses in the European Alps and the Himalaya regions, which are built to take large loads of snow.

The other factor in the shape of our built world is the stylistic tradition that develops in a certain region and culture. A very well known example of the process of developing building style is the shape of the Greek and Roman stone-work temples. Certain aspects of this building typology - for example the entablature (the decoration on the longer side of the temple, above the columns), is rendered in stone as it was rendered in an earlier form of building, that used wooden beams in this place. There's a through line to find in the development of architectural styles from the way a building element is formed due to the technique used to make it, to the use of this element as an ornamental form of a certain style of architecture. I would recommend to read up on Adolf Loos' "Ornament and Crime", an essential text in the development of the modernist style that discusses this formation of ornaments in architecture (and other crafts) and helped paved the way to the very modern notion of "Form follows function". Still, Loos and other early modernists also argued for the development of new ornaments based on the newly developed building technologies of that era - coming full circle to the notion that the form of what is built always depends on the available materials and how we use them.

Thus, the decision to build something in a rectangular or circular (or otherwise) shape, is a combination of the regional conditions, and the developed traditional building culture of a certain time and place. I cannot personnally comment on either of those for the construction of the Egyptian pyramids specifically, but I hope this has given you some insight nontheless.