When you take a step back, business suits (and especially ties) are a very, very strange institution. Why do we have a "grown up" uniform at all? Why does it have this particular form? Why is dangling a piece of cloth from your neck considered anything other than ridiculous?
I've read the existing answers (linked in the FAQ) to questions about suits and ties. They focus on tracing the fashion precursors of the current styles of the suit. For instance tracing contemporary ties to cravats worn by Croatian mercenaries. While this is fascinating, it does not answer my question. I would like to refocus discussion on why the institution of the business suit exists and is so widespread today.
Let me be more specific. Here are the reasons that I find the very existence of business suits counter-intuitive:
(Non-functional) Unlike some uniforms (e.g., hard hat, overalls, flame-retardant suits, and even deliberately revealing "Hooters" uniforms), business suits are entirely non-functional. They are usually worn by office workers, who could do their job just as well (perhaps better) if they were wearing something else.
(Low signalling value) Often the ways people (and animals) choose to dress (or evolve to look) are designed to communicate to others who or what they are. However for a signal to have value (at least in biological evolution, which I'm more familiar with) it must be hard for at least some individuals to fake the signal. However it's easy for anyone to go out and buy a suit or to borrow one from someone else. There are services that loan good quality suits to homeless people going for job interviews. These days a suit provides as much signalling value as a little pin that says "I'm a business person!" Were suits once unattainably expensive, and so had signalling value, but now the price has dropped yet the norm persists?
(Mandatory) Some progressive companies like Google are (apparently) letting their employees come to work in neat casual attire, as long as they get the job done. However, as far as I know, most big companies in large wealthy cities still insist that their workers come dressed in a strange, seemingly arbitrary "business suit" uniform. Where it's not mandatory, my understanding is that workers are often disadvantaged by not wearing one. Given that the uniform is both non-functional and its signalling function is easily faked, the insistence that people must wear them seems especially peculiar.
(Arbitrary) Perhaps there are reasons for the particular normative form of the business suit. For instance, the presence of a tie (forbidden for women), the presence of a pointy collar, the standard colour scheme, skirt/dress style and length for women (forbidden for men), shirts must have long sleeves, etc. However to me they seem entirely arbitrary. Why not, instead, have a short sleeved shirt with collar that goes all the way around the neck, with pieces of coloured cloth hanging down the arms? If there is some historical or contemporary function to the particular form of the business suit, I would love to learn it. To be unambiguous here, I have read existing answers tracing the precursors of the current suit forms; that's not what I'm asking about. What interests me is the arbitrariness of those forms. Are they arbitrary? If so, why should something so arbitrary be so prevalent and so strictly enforced?
(Prevalence) Despite the peculiarities above, the business suit seems to be present worldwide. Business people wear it in India, Brazil, Japan, Kenya and Finland. How did something so seemingly arbitrary and non-functional come to have such widespread adoption? How does it retain its seemingly arbitrary form while migrating between cultures and generations, despite the pressures of innovation, reinvention, cultural assimilation, and historical change that buffet most other domains of human activity (e.g., architecture, decor, language, art and media, etc.)?
And so, historians, I ask you: Why the "Business Suit"?
Why is something so non-functional, with such little signalling value, something whose form is so arbitrary, nonetheless so prevalent across the world and often even mandatory?
So, what you might have noticed in the other discussions of this topic is that fashion isn't random or arbitrary. It's a slow, gradual process that is very natural. It's not guided by individuals as much as the eventual change in aesthetics that comes with any culture changing with time. No one chooses these things. It was somewhat acceptable for women to start wearing trousers for casual circumstances in the 1950s, for example. It wasn't a fashion designer or famous star that caused this to happen in the years just before. We have to go back to the 1840s for the Bloomer suit in Euro-based fashion. It truthfully took that long, with different groups picking up the fashion here and there, for it to catch on enough for the regular woman to consider putting it in her closet. The likely-hood of someone just deciding that they are going to require a short sleeved shirt with color and pieces of coloured cloth hanging down and it actually catching on are pretty much next to impossible. Even if a fashion designer decided this, and managed to get it produced and publicly available (after a few years), who would buy such a thing and be the weird one out at the office? Individuals can't decide fashion, maybe inspire it, but it's a rather large and slow moving creature.
Now, addressing the other parts. Business suits may seem uncomfortable and difficult to function in today, but it wasn't always so. When they were fitted custom to the person their movement was taken in to account so that they could manage to do what they needed without difficulty. However, you'll still see that laborers will wear different clothing that is much easier to do hard work in. Therefore, the suit itself becomes a way of saying you don't do hard labor. You are in a job that is not blue-collar and you can wear a suit where you don't have to do any physical work.
Which leads us into the next part. A suit still says something, and it always has. It's not easy to just end up with suit today, in my opinion. They are so much more expensive than they used to be. Can you borrow one? Sure. Better hope it fits you well because wearing a poorly fit suit can look worse than none at all. And there is where the difference has been for centuries. Not in the title of the clothing item you own, we all have pants, but in the quality of the fabric, stitching, and fit of the item. Most of modern society has lost the ability to judge this well, but in the circles of society where suits are still the norm they can tell if it's off the rack from Men's Warehouse or bespoke from a tailor.
Suits were worn by everyone until the early 20th century, just not for every job. But, this means they have to be affordable. Let's look at the 18th century system because it starts to change a lot in the 19th century, but the for many centuries it was the system. If you want an expensive suit you go and purchase expensive, high quality fabric, lining, interfacings, buttons, and threads from a large store with hundreds of options. You take these supplies to Mr. Durfy, your favorite tailor in town. He charges you 1 pound 3 shillings to make the suit of plain breeches, waistcoat, and coat. If you are not rich and need a cheap suit you go and purchase cheaper fabric, lining, interfacings, buttons, and threads from a large store with hundreds of options. You take these supplies to Mr. Durfy, your favorite tailor in town. He charges you 1 pound 3 shillings to make the suit of plain breeches, waistcoat, and coat. Yes, it costs the same for the labor. Your fabric might have cost you dozens of pounds or less than one. So, everyone gets a well made suit that fits, it's just a very different quality. Suits weren't worn to signal that you were high class. They were standard attire. Now, if you chose to trim it in gold and silver with sparkling buttons and embroidery you're trying to say something. Regularly, though, people would wear a suit appropriate for the season and whatever they would be doing that day.
So, how do we tell if that man wearing a brown broadcloth suit is a gentleman or a tradesperson or even a poor farmer in his best clothes without going up and touching the cloth? By the way he walks, talks, and holds himself. As we progress into the 19th century and the middle class forms, there's a lot of concern in the upper class over the middle class trying to mirror them. "New money" can buy their way up and wear nice clothing, but you can't put lipstick on a pig, to use a phrase. This new middle class will choose to wear the suit for business purposes because it is an apparent way of saying "Look! I'm not wearing clothing for labor! I'm clearly doing well." They still aren't high class, but they can aspire and imitate.
The very recent push towards casual attire in the workplace is something that mirrors not only the general publics view towards clothing, but in the case of Google specifically, a very California view of clothing. Our entire definition of formality has changed, and with it the workplace. The culture of New York City is vastly different than that of Mountain View, CA (or even that of where most of their workers are located at data centers on cheap property around the world). The last thing a business wants is for their staff to be under-dressed because it sends out a signal to their clients or other businesses that they work with that they are lazy or too casual in their approach. Some day it might be that all places stop wearing the suit and go to jeans and a t-shirt, but until those business are no longer trying to get work from people in a suit they'll continue to put forward the effort to show that they are willing and able to put forward effort every day. (P.S. Google does actual require employees to follow a business dress code when working with other businesses that require it for their employees).
As for what you see as arbitrary fashion rules, they aren't. Every thing is part of that gradual growth. So, no short sleeves for example. This is because of the age old idea of "rolling your sleeves up" to do labor. Thus, shortened sleeves are working class and casual. That color scheme is so we aren't being distracted by flashy colors or judged for poor choices that make us look bad (black is safe, calm, clean, and looks good on everyone). The pointed collar was a standing collar for a while, then rounded, etc. It just happens to be the aesthetic that sells right now, so it's what keeps getting produced and is "normal".
The migration of the suit through all cultures comes back around to the earlier topic of business working with clients and other businesses. To be taken seriously by large European corporations traditional dress was set aside by international businesses and the standard suit adopted instead. This will trickle down to other professional businesses that may not have the large international scale, but work on a national level with the big businesses. If someone walks into a board room wearing shorts and a t-shirt or a traditional garment of their country when everyone else in the room is wearing a simple, well-fit grey suit they will be the odd one out. The "other". Regardless of the opinion of whether it is insulting that they were so casual or that they aren't adapting to the culture they are in, that "otherness" will cause them to be seen as an outsider, not part of the group. This does not bode well for respect or cooperative business.