The short answer? There wasn't a general transition.
Military centered swords remained very similar into the 19th century, essentially adding developed hilts to arming sword blades. Now, of course there were many types of arming swords, some were purely thrust-centric designed for defeating armor see this video for an example at the Wallace collection. But for this topic I'm simply referring to the single-handed cut and thrust style sword with a medium length blade. For examples of early modern swords of this type, see the Broadsword and Backsword (essentially a single-edged Broadsword). These were essentially arming sword blades added onto developed hilts. Early modern militaries preferred these as sidearms, and they continued use alongside sabres in the 1800s. See for instance the 1796 Heavy cavalry sword. So the while the rapier did see use as a sidearm in military contexts, the arming sword simply wasn't replaced by it. Heavier, cut and thrust swords were far more common. Indeed, cut and thrust style swords lasted longer than the rapier, which was largely replaced by the smallsword in the early 18th century.
The rapier developed out of the need for nobility to protect themselves outside of military contexts. Contrary to popular conception, the rapier is a very substantial weapon, often weighing as much or more than an arming sword. However, its design lends itself to unarmored combat. The long, relatively thin blade is excellent for piercing relatively light targets, and gives a reach advantage over other swords (it is not uncommon for rapiers to have blades over 40" long, far longer than your typical arming sword). Eventually, as sword-based self-defense became less common, and swords became used less for killing and more for fashion and duels, the rapier gave way to the lighter and less lethal smallsword. So the rapier became the go-to weapon for civilian self-defense because its form lends itself to that purpose.
To sum up, there wasn't a general transition from arming sword to rapier. Rapiers did replace arming swords in civilian contexts, but wide, mid-length, cut and thrust blades remained common weapons of war from the Roman spatha to 19th century cavalry swords.
