It's impossible to generalize this question, unfortunately. The Medieval period was a long time, and there were so many distinct regions that had their own legal and cultural standards for arms ownership that what might be true in one place would be odd or abhorrent in another.

I can give some insight into the late medieval culture of arms in the Holy Roman Empire, however. B. Ann Tlusty has suggested that the mid 16th century was the high point in the ideological and physical connection between public masculinity and arms-bearing.

There's a lot to generalize here but the salient features are these: cities often relied on guilds to provide public services; those guilds orchestrated a complex series of rights and privileges for their members in exchange for duties and services; men who performed these services through the guilds were given the status of citizen (this was not the only way, of course, and citizenship is an amazingly complex idea); one of the privileges of citizenship was the right to carry weapons in exchange for the duty of performing guard duties and firewatch for the city or town.

This was in some ways a breakdown from the earlier three estate conception of society, in which everyone (or at least every man) was sorted into one of three categories: those who Pray, those who Fight, and those who Work. These can be understood not only as categories of social position, but also as an expression of their duties in society. The clergy protect mens' souls, the nobility protect mens' lives, and the peasantry grows the food men need to eat. Without all three of these things, society crumbles.

The guildmen were an odd intrusion into this cozy paradigm. While they do work, they work for profit, not to serve society. But they also provide things like weapons and armor for the nobility, provide credit and reputation networks so that nobles can afford to run their estates and even for the church, they build boats and trade, they in essence do a lot of work that allows the world to function, but because many of them are wealthy - rivalling that of the nobility themselves - they expect privileges commensurate with their value. To make a complicated social and cultural negotiation simple, nobles would often lease or sell portions of their land - typically villages or cities on their property - to the city itself, usually through a guild hierarchy, to make a "free city" which owed typical taxes and fealty to their rulers, but were free to dictate the business of the city on their own. Because the city leadership was all made up of burghers in the guilds, the guilds themselves decided who could be considered a citizen and enjoy the benefit of their social station, and who wasn't.

In short: members of guilds in free cities (and elsewhere, as the idea caught on regardless of the legal structure of your town may have been) were often not only allowed to carry weapons, but required to. Burghers were expected to work, like the third estate, and also to protect themselves, their property, and their city, like the second estate.

This process was ongoing as far back as at least the 14th century, was widespread by the mid to late 15th, and was firmly established by the 16th. Burghers - that is, citizens - were wearing many hats in society, expected the respect and privilege that it earned them, and were required to perform duties and services in exchange. This helped to institute an honor culture among the burghers, reflecting, in a broad sense, the honor culture among the nobility. An honor culture means, in essence, that a man is expected to respond to threats to his person, property, family, and honor with violence. To be expected to respond with violence meant an expectation of being armed, and being skilled with those arms, which led to an explosion in popularity of fencing with many weapons, and civilian-facing fencing demonstrations and competitions, as well as shooting contests with crossbow and musket, and even contests that pitted the artillery of one village or city against another. A man might be expected to own a sidearm (usually a sword), a battlefield weapon (a pike or halberd), and a ranged weapon (a musket or crossbow), and be at least basically proficient with each. In some places there were laws enforced through fines that men could not lend or sell their weapons, meaning that not only was it common that a man would own a sword, it was absolutely required!

Of course, again, this is only true of some Holy Roman Empire cities and towns, but it is emblematic of a specific trend that was sorting out a new estate in medieval and early modern society: where exactly are the boundaries between "those who fight" and "those who work" if there's a new class of man that is neither noble nor peasant, and works as well as fights? What does that mean for society? What does it mean for the peasant, and for the knight?

There were tons of different answers, but guild-sponsored citizenship, and the transfer of the burdens of defense from the nobility to the citizenry was one answer.

---

The best exploration of this topic is in B. Ann Tlusty, The Martial Ethic of Early Modern Germany. Sydney Anglo tackles this topic quite a bit through a number of his works, as well.