Unfortunately, perceptions of social dance throughout history are so influenced by often inaccurate views on the eras involved, so little thought is placed toward historical accuracy, and living traditions play such a part in many modern depictions, that finding even somewhat accurate depictions of historical dances can be very difficult.

While I have limited knowledge of Baroque-era dance, for example, I can say that the “Hornpipe and Rigaudon” you link to seems to bear little resemblence to court dances of the era, and it’s certainly not a Hornpipe! It looks a bit like some sort of choreographed country dance. Similarly, the waltz you show is a performance waltz danced by Stanford, which over the last few decades has developed its own peculiar waltz style.

The shift you notice, however, is there, if not so drastic or fast. One thing to consider is how social and court dances developed and propagated. Sharp (1924) views the development of social dance in the 17th and 18th centuries, for example, as a process of dances being developed, often being taken from folk/country dances and popularized in one region, and then being exported more widely to other countries.

While, again, I’m not quite knowledgeable on this point, Baroque “court dances” like the Menuet, Gavotte, and so on are largely of French construction (see Power’s website, for example). They were, in fact, couples dances: they were arguably the epitome of couples dancing, as many of them were danced by one couple at time. They were highly choreographed, precise, and very much performances by a couple for an audience. They were also constructed dances developed within the court structure, bearing little resemblance to dances of the country.

At least in England, I should note, they seem to have also been wildly unpopular at many subscription dances. The New Bath Guide, which includes rules for the Old and New Assembly Rooms for the second half of the 18th century, has all sorts of rules for “dancing,” like precedence, numbering, and so on that actually applied only to country dances, along with a few special notes about what people who wanted to dance a Minuet should do, including particulars on clothing, and the need to talk to the MC about it beforehand. There, in the 18th century, the Country Dance, originating in England a few centuries before in the countryside and exported elsewhere, was far more popular. These were, by contrast, rather simple improvised (or falsely improvised through memorization) dances where the lead couple would determine the steps, and others would follow them. There was also a performance element, but now on the more limited scale of the set of dancers around you rather than the entire ball. They were, by many accounts, often danced in a rather wild manner, especially outside of court (see Dickens writing about dance in A Christmas Carol, for example, or many of Wilson’s rants about dancing in his dancing manuals and the Danciad).

The waltz, finally, was a dance originating in Germanic regions. The performance and audience element really isn’t there in the same way, and having a very different region of origin, the dance is consequently quite different. While the actual history of its origin is also a bit unclear, it appears certain to have come from rural dancing without a huge period in between, rather than being something that was constructed by dancing masters.

Powers (same site) argues that it was the French revolution that brought about the demise of the older court dances, and thus a vacuum that allowed the dissemination of the waltz to the rest of Europe. I might speculate that the French style of dancing the waltz in the first years of the 19th century, which was quite shocking to many, may also be related (see Wilson’s waltz book).

And the waltz was quite shocking. That Wilson, writing descriptions of the waltz for an English audience presumably in an attempt to promote it, spends more time arguing about how the dance should not be viewed with disapproval than he spends talking about the actual dance is rather telling. Even Byron, famously, thought it rather scandalous. It arguably was. When it first rose to popularity, it was not the dance we see today, or even the waltz of the 1840s or 50s. The Regency-era waltz, especially in the French style, did have very close, flexible, vaguely defined holds that could be seen as being rather odd even today (Wilson has some drawings). There were disagreements about the acceptability of the dance for many years, but at the same time, the form of the dance changed to become more rigidly defined. Compare Wilson to Hillgrove a few decades later, for example, and the difference is clear: far from the French waltz in Wilson that pretty much involved wrapping arms around each other in vaguely defined “attitudes” and going around in tiny circles, the waltz of the mid 19th-century had a hold that was well defined and inflexible and precisely defined steps.

I would suspect that this change in the form of the dance from flexible and informal to somewhat more rigid and formal made it more acceptable, though I need to do more research on this point. This can, however, be seen to some extent today: modern competitive ballroom styles often have far more skin contact than many other styles of dancing, and yet, being far more formalized and rigid, are seen as more acceptable.

Still, the waltz was not a court dance, and didn’t really become one until much later, perhaps some time in the 20th century, when courts in general were very much in decline. Round dances in general (waltz, polka, galop, schottische, etc) were never really seen as court dances, and I would speculate that this had to do with the lack of the performance aspect, combined with the decline of court-style dancing in general.

The quadrille largely took on the role of a court dance through the 19th century, and to some extent, had a performance aspect, especially in (rather rare, to my knowledge) situations where only one set danced (the ball scene in Trollope’s Way We Live Now mentions this, and Giraudet’s Traite de la Danse has some quadrilles that seem intended for this). At state/court/ceremonial/etc balls in the late 19th century, the order of dance often shows an absurd number of quadrilles, with dancing often alternating between a round dance, which varied throughout the evening, and the same two quadrilles over and over again. Sharp suggests that this really doesn’t seem like something that would have been popular, while one dancing master in the mid-19th century suggests that quadrilles were in such decline it was often difficult to assemble even one set of four couples at a ball. I’m also particularly reminded of several instances in contemporary fiction of the late 19th century where dancers particularly avoid quadrilles. Interestingly, in The Quadrille (2000?), Rogers describes the quadrille at the time as becoming, at the same time it was used as an unpopular ceremonial dance, a dance of the middle and lower classes, with the complex steps largely ignored and the whole thing made comparatively wild and jovial.

Thus it might be argued that the waltz, and other round dances, served as dances that were not seen as lower class, but also not stifled by ceremonial use. They had somewhat more contact, yes, but after the unusual forms of the first few years of popularity, they were codified into something significantly more acceptable, with less contact. As court dances of the Baroque era were largely danced by individual couples, the change was not too great, and I would argue that the lack of the performance aspect (and decline of courts in general) was the larger barrier to the waltz as a court dance.

The comparison to the Country Dances popular in England was a different case, and there were, I believe people who took issue with the coupling and the individual attention it allowed. However, country dances involved being with one partner, if in a group, for far longer periods of time (Susan de Guardiola speaks of this on her website), and arguably offered far more time to converse and lavish attention on one’s partner.

I realize that in writing all of this, I haven’t answered your question very well, partially because my knowledge of the social changes surrounding the changes in dance are rather limited compared to my knowledge of the dances themselves. But hopefully this gives some slight insight into the changes: I honestly do think the biggest factor in what the shift seems like has to do with the wild inaccuracy of many modern reenactments.