Around the same time as cotton jersey (the material they’re made from) and the knitting machine were invented. The knitting machine dates to 1589 when it was invented by William Lee (1) and by 1598 it had been refined enough to knit finer fibres like silk, to make silk stockings, cotton stockings, and wool. His assistant John Ashton added a divider mechanism that facilitated the process after Lee’s death.

Later, in the 18th century many companies were working to refine the knitting machine to knit with silk, cotton and wool. One of them was the Derby rib, where knits and purls (backwards knit stitch) are alternated, which creates a fabric even more elastic than regular knit. This is just like that stretchy fabric that forms the wrist bands and bottom of clothes like sweatshirts.

T-shirts could have been made as early as 1844 according to this website (2), here’s a knitting machine frame from that era, which knit wide pieces of fabric which would be cut down to make t-shirts, and other underwear. Here’s an image of men’s underwear (3) from that era, made of the same fabric as a t-shirt. The fabric was called Jersey, because it was an export from Jersey in the Channel Islands, but there’s nothing in my skim of sites that attributes that name to more concrete findings.

In 1916, Coco Chanel started to use cotton jersey in exterior garments like sweaters, dresses and the like. Keep in mind, then, men wore what we know as tank tops and t-shirts as undershirts, so it would have been viewed as making clothing from a scandalous fabric.

1- http://www.historyofinformation.com/expanded.php?id=3634

2- http://www.knittingtogether.org.uk/docEX35a5.html?doc=13661&cat=738

3- http://www.knittingtogether.org.uk/docEX3a04.html?doc=13678&cat=740