First, swords were not prohibitively expensive by then. Even as early as the 13th century, there were relatively affordable swords. For example, one sword stolen in 1281, in England, was valued at 9d (ref [1], pg 43), approximately the cost of a gallon of good wine, a pair of boots, a week's wages for a labourer. Also in [1] (pg 209), we find swords for under 3s in the mid-16th century, approximately 1 week's wages [2].

Second, just because you can afford a sword, or own a sword, you don't necessarily wear it everywhere. Artwork can be a good source for what was worn as everyday-carry weapons. Daggers and knives are common, and some of the knives are very large. Some example from art:

• 1560s: https://commons.wikimedia.org/wiki/File:Pieter_Bruegel_the_Elder_-Peasant_Wedding-_Google_Art_Project.jpg - the left-hand standing musician has a large knife, and the well-dressed man on the far right has a large and expensive-looking dagger. Other knives can be seen.

• 1560s: https://commons.wikimedia.org/wiki/File:Pieter_Bruegel_de_Oude_-De_bruiloft_dans(Detroit).jpg - many daggers/knives; the musician at the far right has a very large knife.

• 1560s: https://commons.wikimedia.org/wiki/File:Pieter_Bruegel_the_Elder_-The_Peasant_Dance-_WGA3499.jpg - large knives.

• 1550s: https://commons.wikimedia.org/wiki/File:Pieter_Aertsen,The_Egg_Dance(1552).jpg - daggers are worn, and there is a large two-handed "knife" on the floor with a large Katzbalger-style S-guard.

• 1615: https://commons.wikimedia.org/wiki/File:Pieter_BRUEGHEL_Ii_-The_tax-collector%27s_office-_Google_Art_Project.jpg - no prominent knives/daggers, but most of the men are carrying sticks, which can be used as effective self-defence weapons.

• 1610: https://commons.wikimedia.org/wiki/File:Pieter_Brueghel_II_-_A_country_brawl.jpg - agricultural implements can make effective weapons: flail and fork in a brawl.

Tlusty [4] covers the ownership and use of arms, including swords, in Germany. Swords were common (e.g., see table 5.7, of arrests for weapons-related incidents, by type of weapon; swords are a large fraction, and the single largest group of incidents; the totals are: swords 125, knives & daggers 83, polearms 8, firearms 40, crossbows 2, blunt weapons 31, other/unknown 11). Moving into the 18th century, swords became more upper-class, and the middle/lower classes were more likely to fight with canes/sticks than swords. Tlusty gives further artistic examples of swords. Comparing Tlusty's information and art for Germany with the Dutch art examples above, swords were more common in Germany, more likely to be worn by peasants, and the transition from sword to cane/stick might have occurred earlier in the Netherlands. However, while many peasants in Germany in the 16th century owned and wore swords, the majority of peasants in Germany did not. While swords are more common in German art, they are worn by a minority of peasants, and Tlusty (pg 171) notes an arms confiscation that reported 16% of peasants had swords. For peasants, knife and dagger (and stick, and farm tools if at hand) sufficed for self-defence (and polearms and guns, and armour, when preparing for war). Swords were more common in cities.

[1] Charles Welch, History of the Cutlers' Company of London, The Cutler's Company, 1916. https://archive.org/details/historyofcutlers01welcuoft

[2] Jan Luiten van Zanden, "Wages and the cost of living in Southern England (London) 1450-1700", http://www.iisg.nl/hpw/dover.php

[3] James P. Ward, "Prices of Weapons and Munitions in Early Sixteenth Century Holland during the Guelder's War", J. EUR. ECON. HIST. 33, 585 (2004)

[4] B. Ann Tlusty, The Martial Ethic in Early Modern Germany: Civic Duty and the Right of Arms, Palgrave Macmillan, 2011.