You're right, this is a really broad question, but since you mentioned GoT, I'm going to narrow the focus to the approximate era that (sort of, in a highly fantasied way) GoT is based on. Since GRRM has mentioned that he borrowed heavily from the period known as the Wars of the Roses, I'll say we're dealing with 14th-15th century Western Europe, and go from there. The reason I'm not going to deal specifically with English fashion is mainly because we don't have much in the way of extant clothing from this period without piecing it together from other meagre finds elsewhere in Europe. Especially with regards to "peasant" clothing.

What we do know, based on the bits and pieces from bog sites around Northern Europe and Ireland, is that medieval people had a high degree of sophistication when it came to understanding the geometry that goes into tailoring a flat piece of fabric into a 3-dimensional garment, so right off the bat, the "burlap sack" movie trope falls apart... at the seams... Sorry.

We also know that fashion was more extreme at the top of society than it was at the bottom, but that generally speaking, the fashionable silhouette remained similar across the social spectrum. Consideration had to be given to the amount of mobility the wearer needed, with a wealthy person who doesn't do much, if any, manual labor having the luxury of very fitted, restrictive clothing (excluding clothing worn for hunting or military pursuits). This is compared to a working person, who needed clothes that they could move around in. But like I said, the general silhouette holds across classes, so if body-conforming clothing was en vogue for the rich, looser fitted, but similarly shaped clothing was preferred by the lower classes. One really only has to look to the Labors of the Months in the Très Riches Heures of the Duc de Berry to get a pretty good idea that the lower classes didn't look like Dennis in Monty Python's Holy Grail.

Pour example: Sheep shearing in July, Haymaking in June, Planting fields in October, and so on.

As for fabrics, there was linen (almost universally used for the layers that came into contact with the skin) and there was wool (all the other layers). The fabrics were likely coarser than what the elite were wearing, but that's a given. Fabrics could be dyed pretty much any color you could get from a plant or mineral that would have been locally available. Colors obtained from indigo, murex, and kermes were more expensive to obtain and maintain, so those were off limits more due to one's income level than anything. And even relatively primitive fabrics could be woven in an astonishing array of plaids and twills, so a commoner wouldn't be limited to just dirt-colored clothing in plain weave like we see in the movies.

Edit: I forgot to mention that, especially in the case of the bog clothing finds, the ubiquitous brown color of the fabrics is a result of having been, well, in a bog for several hundred years, and may not reflect the original color of the garment. If we look to illuminated marginalia (which has a whole bunch of caveats attached), we at least see that the garments were an entire range of colors.

Hope that answers some of your questions!