A question that’s interested me for a while (in the title). I know enough about the music itself as I’m a musician but I don’t know that much about what was really going on during that post-renaissance period.
EDIT: a word
This is rather a simple question. In the Renaissance, dances began to be done as a medley. A pavane ( where anyone of any age could dance, in a stately way) might be followed by a galliard, which would be much livelier. Sometimes the same theme for a dance in 4 would be played in 3 for a Nachtanz. That simply grew into a musical form for musicians to play in concert: the dance suite. By the Baroque , after 1650, what had been introductory music to get dancers on to the floor and formed up had become the prelude, which would be followed by galliards, courantes, pavanes, allemandes, bourrées, etc. By the Baroque some of the older dances had become pretty much only concert items- no one danced the galliard, in J.S. Bach's time, and it was played much more slowly than it was when people were actually dancing it ( and , for that matter his famous Bourrée in E Minor is great in concert but pretty wooden for actually dancing one), and the Passacaglia became much more common as complex variations over a theme in the bass. Still, some composers wrote dance suites that could actually be used for dancing, like Rameau, or Bodin de Boismortier.
In the later 1700's the form became less popular for concerts, supplanted by concertos and sonatas. But of course it's been sometimes revived : Ravel's Tombeau de Couperin, Stravinsky's Pulcinella, are dance suites. And dance forms lurk within bigger classical compositions; for the jiggety 2nd movement of Beethoven's 9th symphony the sheet music says scherzo, but his listeners circa 1830 would have quickly imagined a lively little quarrée or square dance.
And if you look at traditional music, a similar process is happening. Not limited to playing for a dance, an Irish band in concert will play medleys of dances, often a couple of jigs followed by a few reels. Scottish fiddlers practice "MSR", or March-Strathspey-Reel. It is, really, a natural thing for a dance musician to play medleys of dances for a concert.