This is intended to be in asked in a historical context. The development of clothing in regards to its professional appearance and concern for comfort. Double edit: Originally asked referencing Western culture, but thinking about it now, reading about Eastern culture would be cool too.
The simplest way to explain it is that even as recently as the early 19th century almost all fitted clothing was custom made. This means that not only will it fit the person well, but it will be designed to move however they need to. A jacket fit to the same person for dancing, fencing, blacksmithing, or just standing around looking good will all be vastly different. In the case of sleep wear, sleeping shirts and shifts were loose and unfitted garments since appearance was of no concern in these situations. Those were most often mass-produced items. Clothing has continued to develop with the persons actions in mind, formal clothing assuming you will not being doing any major movement. Add in that formal wear for most people today is not custom fit makes it potentially restrictive and uncomfortable. Men's coats are a perfect example of this; the arm hole in ready-wear coats today is very large. They'll claim it's for a roomier fit, but in reality it's because it makes it much easier to fit a random size. What this does instead is prevent the wearer from lifting their arms to the side without greatly disturbing the body of the coat, a non-issue in earlier centuries where the tailor fitted the arm hole closer up under the arm to allow for easy movement. A custom-fit suit or gown made today will be comfortable because it, again, takes in to consideration not only the persons body shape/size but how that body will need to move. Even if it is only to be able to drive the car easily.