The short answer is that fabric armour (gambesons, jacks, doublets of fence - there are a number of different garments with different names) is indeed very effective, even when used as a standalone armour. But it isn’t as effective as plate armour. The long answer, though, is much more interesting but more nuanced.

First of all, what was fabric armour? There are a variety of fabric armours throughout the Middle Ages. They differ in cut and construction, and terminology changes with time. but are broadly made in two ways - they are layered or stuffed. Stuffed armours are padded with rags, raw cotton, tow or other materials described by the technical term ‘fluffy stuff’. Layered fabric armours are layers and layers of fabric lain on top of each other and generally quilted together. Fabric armours could be used as a supplement to armour - under or over other types of armour. It could also be with combined with other armours as in the jacks ‘stuffed with horn’ or ‘stuffed with mail’ in the Fastolf accounts of mid-15th century England, or possibly ‘jazzerant mail’ (depending on weather that included padding or just fabric coverings and linings) But since you asked about standalone armours, let’s talk about those.

In the 15th century the main form of standalone fabric armour was the jack. It is shorter and somewhat more fitted than the earlier gambeson, based on illustrations - neither 15th century jacks nor 13th century gambesons survive. They are the most common armorurs in the mid-15th century Bridport muster roll (an English muster of local militia, not first-line soldiers) by a factor of 10, and attested as being incredibly common among the archers of Richard III by the Italian traveller Mancini. Importantly, these were archers who would expect to face other archers in battle - with the Wars of the Roses flairing now and again (they ended at Bosworth a few years after Mancini visited) English armies were used to fighting each other, with large numbers of archers on either side. King Louis XI of France laid out the construction of a layered jack in his ordinances. The author/sometime-reenactor/museum exhibit designer Gerry Embleton translates the French thus:

"And first they must for the said jacks, 30 or at least 25 folds of cloth and a stags skin; those of 30, with the stag's skin, being the best cloth that has been worn and rendered flexible, is best for this purpose, and these jacks should be made in four quarters. The sleeves should be as strong as the body, with the exception of the leather, and the arm-hole of the sleeve must be large, which arm-hole should be placed near the collar, not on the bone of the shoulder, that it must be broad under the armpit and full under the arm, sufficiently ample and large on the sides below. The collar should be like the rest of the jack, but not made too high behind, to allow room for the sallet. This jack should be laced in front, and under the opening must be a hanging piece (porte piece) of the same strength as the jack itself. Thus the jack will be secure and easy, provided that there be a pourpoint without sleeves or collar of two folds of cloth, that shall be only four fingers broad on the shoulder; to which pourpoint shall be attached the chausses. Thus shall the wearer float, as it were, within his jack an be at his ease; for never have been seen half a dozen men killed by stabs or arrow wounds in such jacks, particularly if they be troops accustomed to fighting."

The eye-catching claim here is the last - that wearers of jacks were nearly immune to arrows and stabbing attacks. But do these claims hold up? The simplest argument is that if cloth armours didn't work, they wouldn't have worn it. As we see in the early modern period, soldiers become more reluctant to wear armour when it confers only a marginal advantage against some attacks. While they look light and fluffy, gambesons, jacks and other fabric armours are fairly substantial - armour made of layered fabric (like a layered jack) is quite heavy, and it's all fairly hot. It can be somewhat hard to move in as well, due to the thick fabric around the joints (though making the armour thinner around the joints can do this - you can see this on the St Ursula Shrine by Memling. Armour has to provide some significant advantages to be worth those disadvantages.

But while it is a start (of a sort) saying ‘if it didn’t work they wouldn't have used it’ is neither definitive nor very illuminating. Fortunately, it is not our only evidence.

I’ll start with modern tests, because you mention those. Experimental archaeology has its place, but tests need to be tightly controlled, measured and documented to be valuable to us - you are right to be skeptical of the evidence provided by YouTube videos. Unfortunately, the most well-documented tests on armour (those of Allan Williams in The Knight and the Blast Furnace) don't deal with fabric armour in detail. He combines it with his tests with mail in ways that are confusing and contradictory (see my exchange with u/Hergrim). Williams performs only one test on a standalone 30-layer jack - a simulated sword blow, which failed to penetrate event with 200 joules of energy. The '30 layer' figure is important because this is the number of layers stipulated for a standalone jack by the ordinances of King Louis XI in the later 15th century. This is pretty good evidence that jacks provide good protection against swords - Williams quotes swords as providing no more than 120 joules in a slashing blow. So they are sword proof, and proof against many axe blows. What about arrows? Unfortunately Allan Williams doesn’t give a value for ‘arrow’ attacks against armour, nor ‘lance’ attacks, and since he is measuring penetration he doesn’t address blunt force trauma. Williams did test the ‘padding’ (a fabric armour with only 16 layers) against lances and found that this thinner fabric armour was penetrated with 50 joules of energy, and sword blows penetrate the padding with 80 joules of energy. Judging by the difference in sword tests, the 30 layer jack is more than twice as resistant to penetration than the 30 layer jack. If the relative effectiveness holds for a simulated lance strike (a big if) we would expect the 30 layer jack to be penetrated at around 125 joules of energy. Unfortunately, he conducted simulated arrow-strikes against neither. Looking at other armours (the buff leather) we might expect less than 100 joules of energy to penetrate, but that’s pure guesswork. Strickland and Hardy give longbow arrows as having between 80 and 120+ joules of energy at impact, depending upon range.

However, what we can say is that plate armour was much more effective than this, and that there are far more powerful dangers on the late medieval battlefield than swords and longbow arrows. The spikes of polearms concentrated more force in a smaller area than the blades of swords or polearms, couched lances provided tremendous and extremely concentrated force (over 300 joules in Tobias Capwell’s tests) and early firearms were more powerful still. Plate armour offered varying degrees of protection against these (though never total protection); cloth armour couldn’t compete when it came to protecting against these more powerful weapons. The main conclusion to draw is that more testing and study is needed. If anyone is aware of a more definitive and well-documented study of attacks against cloth armour, please let me know. However, we shouldn’t dismiss other, non-experimental sources. Looking at the claims in Louis’s ordinances critically, we see reason to doubt them - of course a king mandating uncomfortable armour for his soldiers would oversell its effectiveness. But still, the claim was made, and made on behalf of an armour that was used for hundreds of years. Based on the incomplete evidence, while it may be overstated, we should take its more general claims seriously.

The examples given - cutting, blunt weapons, and arrows - makes me wonder about one thing: stabbing. Cutting is not particularly effective as I understand it, armor or no. But spears, and to a lesser extent, swords, are stabbing weapons.

How would a gambeson fare against that?

Apart from the Mike Loades tests, what other tests have you seen? Their methodology might shed some light on the results.

All medieval enthusiasts treat gambeson as if it was hands of the divine enveloping you in an aura of immortality. What perplexes me more is, that though their tests are not scientific... they do seem to indicate they were just as good as mail or plate. Cuts? No problem, Mr. Pulovermail has you covered. Blunt weapons? Your armor is literally fluff, you are alright.

One thing that you will see repeatedly is that cuts require more energy than thrusts to penetrate non-metallic armour. As an example, look at the test linked to by /u/dandan_noodles: most swords which penetrated the 30 layer jack could not cut it. The katana, which performed the best a cutting, was used in a different manner to the European swords, performing a tip draw cut that was in line with ideal usage historically. The European swords were still used in draw cuts, but not to the same extend and not with the tip. For another example of thrusting working much better than cutting, look at this video by Skallagrim.

Why is cutting less effective than thrusting? I would say that there are several reasons. Most importantly, I think, is the area the blow is spread over. With a cut, the force of the impact is spread out over the entire length of the blade that is in contract with the textile. The edges also have to cut through much more material with a cut. With a thrust, though, all the energy is focused on a smaller area, and there is less material that the edges need to cut.

As to their real world performance, as /u/WARitter has said, the people who used them clearly believed they offered some protection, or at least said they did. In the case of Louis XI's jacks, I wonder if some form of supplemental armour might have been worn. One experiment has shown that even a relatively weak bow, so far as warbows go, could defeat a jack designed along the lines of the Ordinance, and the author of the Ordinance has "seen half a dozen men killed by stabs or arrow wounds in such jacks, particularly if they be troops accustomed to fighting." (emphasis mine)

It seems as though there is a conditional element to the protection, wherein experienced soldiers will have some extra knowledge that increases it. This could be as a simple as experienced soldiers having the necessary skills to avoid taking many wounds, or it could mean that troops accustomed to fighting know to add an extra layer of defence to the jack. This isn't without some precedent: in the late 13th century a gambeson paired with an aketon was considered roughly equal to metal armour.

On the other hand, the effectiveness of the French jack could just be down to arrowheads more orientated towards penetrating plate armour and the thin padding underneath, at the expense of being able to penetrate dense textile armour.

The important thing to remember is that technology and practices changed a lot, and we don't always know why or if they even changed at all, just the terminology. Some armours performed well in their periods, but might not have functioned well outside. The felt armour of the late 12th century, for example, appears to have performed very well against bodkin heads. Did it fall out of use because linen does better against cutting edges than felt, or did linen become more economical than that quantity of felt? We don't know, and I don't know that any proper tests have been done to test the first, and doubt there is enough evidence to test the second.

So, did textile armour perform well or not? I think that the answer is that it did, but metal armour always offered better protection for a given weight. If a gambeson was god tier armour, then plate would never have been developed, as you say. On the other hand, it was still reasonably expensive and we know that there were forms of textile armour which were considered adequate standalone armour. I doubt people would have bought something that was worthless as protection, even if they would tolerate the weight and discomfort, which indicates to me that there was indeed some value - just perhaps not as much as the tests you have seen have led you to believe.