That's a topic that is, to say the very least, a deep rabbit hole. There are large Masonic research bodies that have poured over every historical document they can get their hands on for, literally, centuries, and we still don't have a complete answer. What I'm about to say is a melange of the output of a few authors ranging from Mackey to Jones, all of whom tend to keep the more grandiose speculation out of their writing...

First: context.

In the West, early Greek philosophy, Jewish philosophy, early scientific work and Catholic metaphysical speculation all merged in a period we call the enlightenment, more or less during the 16th and 17th centuries (depending on which region of Europe you're talking about).

Also during this time and before, a system of labor unions had developed around stonemasonry among other crafts. These unions standardized work and managed the importing of laborers for larger scale construction. They also developed a system of secret initiation by which members would learn their craft and learn to introduce themselves as worthy craftsmen to other members of the guild throughout several nations.

Second: speculative masonry and Grand Lodges.

In this environment of intense philosophical churn and within organizations which regularly interacted with external cultures... something happened. Traditionally we say that the tools and practices of the stonemasons were adapted to a more general philosophical endeavour, but it could have been that an external group adapted both strains into their own work.

What came to be was a system where some of the practices of the stonemasons were merged with some of the philosophical ideas of the enlightenment, and placed in a shell of symbolism and allegory, mostly drawing from the Old Testament as a source for this allegory and symbolism, but not as an element of religious practice (it's important to note that Freemasonry is not a religion). This, we call speculative masonry and it truly becomes "Freemasonry" as we know it today in the early 18th century when the United Grand Lodge of England was formed to regulate these practices and keep them in line with a central set of principles outlined by James Anderson's Constitutions in 1723.

During the rest of the 18th century, the Fraternity spread outward, forming new, wholly autonomous Grand Lodges which operated through a system of charters and mutual recognition.

Finally: appendant bodies.

From the mid 18th century through to the mid 19th century, a set of "bodies" of Freemasonry were established, codifying the most common core of the Fraternity as the "Blue Lodge" and its system of initiation as the universal first three degrees of Freemasonry, the end product being a Master Mason who was recognized as an equal throughout the world, in the Jurisdiction of every mutually recognized (or "regular") Grand Lodge.

The other bodies require attaining the degree of Master Mason before joining, and each carry on a tradition of Masonic initiation that developed in parallel with the codification of the three degrees. Of these, two are still prominent: the York Rite and the Scottish Rite.

The York Rite teaches as set of 13 core degrees in 3 individual bodies that culminate in a Christian-specific body known as the Knights Templar (but are related only symbolically to the historical Knights Templar), but the York Rite itself is not restricted to Christians, only that one final body and its three degrees.

The Scottish Rite teaches a set of 29 degrees (plus 1 honorary degree which is rewarded to a few individuals who exemplify its teachings) which are presented in a dramatic format similar to historical morality plays, and though it is often conferred in brief events (ranging from one day classes to 1-2 year events) the original conferral of its degrees during the 19th and early 20th centuries could take a sizable fraction of a lifetime!