The colonies were mostly rural, sparsely populated. While you mention businessmen and merchants and city police, outside of the few population centers like Boston or Philadelphia mostly what you would find were farmers. The fact that there are records of quite a large number of gunsmiths, the large number of guns listed in inventories in wills, and the pretty large number of surviving guns, provides good evidence that very many of those farmers had guns. In a way this seems odd, as guns ( like most manufactured things) were expensive in the pre-indistrial world. Back in 2000, historian Michael A. Bellesiles tried to use that fact to assert that guns were actually beyond the means of most people: but what it really means is there was a considerable range in quality, from George Washington's fine imported fowler to the very basic trade gun in the hands of one of the luckless pioneers who were illegally trying to homestead on his lands in western Pennsylvania. There was also a lot of repair and re-use: many of those gunsmiths were patching and re-stocking old guns. And there might have been some very battered, very old ones around: mention of a "trumpet-muzzle gun" in a will of 1705 might indicate a blunderbuss...but it could also indicate a matchlock (or some of a matchlock) from 1590.

These were most all for hunting, as you could expect. Hunting in the colonies was not restricted to the wealthier landowners , the way it was in England. But there were also militias, and these would sometimes contract to have military muskets made- now generally called Committee of Safety muskets. Not nearly enough to fight a Revolutionary War, it would turn out, but not too uncommon. And if you were a merchant, had significant assets in the house, you might invest in a pair of pistols in a case, or one less expensive screw-barrel pocket pistol. But those were not that common. Merchants needed a population center, a market, and in many places that really didn't exist: Williamsburg, in Virginia, was a small market town surrounded by large farms and plantations.

How people regarded guns then has been made into a political question now, so you have no doubt seen quite a lot of very broad assertions. But, just trying to stick with the sources, I think it's very safe to say that, then as now, farmers tended to like guns. When they thought it worthwhile, had the money, they sometimes would pay for very nice ones ( or, what we would call very nice ones- compared to what was being made in England or Germany, they were just pretty good). There was also a frontier, and that frontier could be violent- the defeat and displacement of Native Nations was a violent affair, after all, and the frontier settlers were quite quarrelsome among themselves as well. It would be a mistake to say that each of them regarded his rifle as just a tool, in the same way he regarded a froe or an axe. Clearly, a nice rifle could be a prestige item, and a gun could be a symbol of power and security for many living a very precarious existence. When and how that symbol of power and prestige became considered by many to be a Pillar of Democracy and Bulwark of the Constitution- well, I won't even try to go there. But it was not that for much of the 18th c.

Brown, M. L. (1980). Firearms in Colonial America: The Impact on History and Technology 1492–1792 (First Edition). Smithsonian.

The shortest answer is the militia. Service in the militia meant certain men - typically citizens, which meant property-owning members of the local community, and also often meant white property-owning members of the local community, but not always - were required by law to own a "stand of arms," a musket of whatever type the local militia organization required, a bayonet, and a cartridge box. Men may also have to keep and wear a uniform of a specific cut and color, with a specific hat. I have written about this dynamic frequently: where one might get a firelock during the troubles, about the organization of the militia in the colonies and early US states, and on the notion of "well regulated".

Outside of specific embodiment for militia purposes, which would include mustering for emergencies as well as semi-regular drill, a man might carry a firelock or other weapon for hunting, coachmen or stagecoach drivers might keep a firelock with them in case of attempted robbery, duelists would have pistols on the field of honor (but would be transported to and from in a box) and... that's more or less it. Most of the times men would be visibly armed would be while they were mustered as part of the militia, because the militia also served as most of what we'd consider "first responders," today, and early American and colonial "law enforcement" was considerably different than today. Generally, peace officers (a more accurate term than law enforcement, because enforcement wasn't really the goal; customary peace officers were almost entirely reactionary) worked within and alongside citizens, most of whom, again, were members of the local militia. They might arm up for apprehending criminals, but generally your town constables or thief-catchers wouldn't be armed while "on duty," they'd only grab a firelock if they felt they needed on.

This was more or less the same dynamic in towns and cities. Everyone eligible for militia service and not excused for some reason was legally required to own weapons that met a military standard, which mostly meant the musket could mount a bayonet. Some militia companies used rifles, which didn't, but they were still uniformed and required a cartridge box and the company as a whole had a chain of command.

As a whole, militias tended to be somewhat variable. Militia service was more enthusiastic and necessary in western territories, like the Old Northwest and the frontier of Kentucky, because the likelihood of trouble between the regions indigenous population and the settlers was more likely than in, say, Boston. But then in Boston, politics could stir up rebellious sentiments and encourage more men to become involved in organizations like the militia, if the official sanctioned militia was unfriendly to whatever the cause was. In the troubles of the early 1770s, committees of safety and correspondence purchased and distributed arms and ammunition illegally, as the legal sanctioned body of the militia was only to be employed by the governor.

Following the War for Independence, technically illegal but highly organized rebels formed unsanctioned militias during Shays's Rebellion and the Whiskey Rebellion, and both were confronted with militias mostly formed and marched in from out of state. But this shows that organization along the lines of local military companies was a common form of protest and political agitation, not something forced on them from above.

Militias could sometimes be used to empower minorities, as well. In Detroit, in the years leading to the War of 1812, a company of militia made up of former slaves formed under Captain Peter Denison, himself an ex-slave, and was sanctioned by territorial governor William Hull (though challenged by other members of the Detroit government). They saw service in the early Michigan campaign, and their presence was commented on by Ohio militia. They would certainly have been armed and uniformed.

Outside of militia organization, there were a few incidents that look a bit more like the kind of gun-carrying that goes on today. One was the shooting of Charles Austin by Thomas Selfridge. The two had been having a public dispute for some time, and at one point a rumor was widely known that Austin had hired a goon to give Selfridge a thrashing, and so Selfridge armed himself with a pair of pistols. Austin, dispensing with the goon (if he had ever actually hired him) confronted Selfridge on the street with a stout hickory cane, and Selfridge shot him to death. It became a notorious, high-profile court case, eventually decided in Selfridge's favor. But that was highly unusual in this period, and most men would not have carried pistols, even for self defense.

To make a long story short, walking around open carrying like is common today would have been very unusual in the early republic and colonial periods. Being armed was considered part of an organized communal responsibility, and not something done individually, unless it was for some specific purpose, like hunting. Even duelists would not arm themselves until the moments before they fired, and cases like Austin and Selfridge were rare enough that they became notorious.

---

I'd highly recommend Saul Cornell's A Well Regulated Militia for more on arms-bearing in the early republic period.