This is a great question! I read Tolkien in 1967, which was key in inspiring my fifty years of folklore studies - and yet I never considered this question.

With that background, my immediate inclination is to say that a menacing giant spider was Tolkien's creation. Nothing leaps to mind.

The go-to source to answer this sort of enquiry is Stith Thompson's six-volume work, "Motif-Index of Folk-Literature" (1955). Although the publication attempted to be international, it weighs heavily on material from Europe to India - the material that Tolkien would most likely have known (especially when it comes to the Northern European part).

Thompson's index reveals several candidates for a prototype for Shelob (large menacing spiders). This includes Motif B16.6.4, Devastating Spider (India); Motif G303.3.3.4.2, devil as a spider hangs from clouds (Scotland and Wales); B873.3, giant spider (Buddhist myth); B93, Mythic spider (India).

This list is singularly unsatisfactory. The Motif Index has its limits beyond geography; without pursuing each example, it is difficult to see how it is expressed. That said, the examples present in Thompson's work do not seem very Shelob-like to me, and it is particularly difficult to imagine an Asian source serving as Tolkien's inspiration. The Scottish/Welsh example seems to have little in common with Shelob.

Early on, Tolkien was experimenting with various enormous animals to serve as opponents to his protagonists. Given that large spiders appear in his early work, The Hobbit, this may have been part of that idea, something that he expanded with an even larger spider in The Lord of the Rings. But that is only speculation on my part. While I can address folkloric prototypes that may have inspired Tolkien, I can't address his literary sources or the literary growth of his body of work.

Perhaps a Tolkien authority can enter in to answer that important part of the question; all I can provide is the folklore angle.

Edit: Thanks for the awards; much appreciated!

While neither a Tolkien expert nor a historian, nor even a liberal arts major (gasp), I feel I may have something to add this conversation. And so, fortified by a draught of holiday ale, I shall proceed.

As others have noted, Tolkien was a professor of literature, a medievalist, and a life long lover of fantasy. As such, he had access to a great store of knowledge that could have served as inspiration for the now-iconic Shelob monster. For his part, I am not aware of Tolkien ever having acknowledged a direct source for Shelob. (However it should be mentioned that along a similar vein, he famously denied his 'one ring' being influenced by Wagner's Der Ring Des Nibelungen - though, if this be true, they at least drew from the same sources such as the Volsung Saga, and thence, to my mind, Plato's Ring of Gyges.) But I digress.

And, as others have noted, there does not seem to be a good candidate for Shelob in any of Tolkien's dark age or medieval northern European sources. So that leaves us with more modern fiction and fantasy that he may have drawn from. From my own personal readings, a few sources come to mind that A.) Tolkien was aware of and probably/definitely had read at some point and B.) Seem to be sufficiently similar to Shelob as to warrant mention.

The first is the fiction of Robert E. Howard, an American author whose Conan the Barbarian stories preceded Tolkien and whose work Tolkien was familiar with. In Howard's short story The Tower of the Elephant, the hero Conan has burgled his way into a chamber at the top of a sinister temple and nearly meets his end at the hands of a giant spider monster.

Springing back, sword high, he saw the horror strike the floor, wheel and scuttle toward him with appalling speed - a gigantic black spider, such as men see only in nightmare dreams.

It was as large as a pig, and its eight thick hairy legs drove its ogreish body over the floor at a headlong pace; its four evilly gleaming eyes shone with a horrible intelligence, and its fans dripped venom that Conan knew, from the burning of shoulder where only a few drops had splashed as the thing struck and missed, was laden with swift death.

Let's move next to another author Tolkien acknowledged as an influence. In George MacDonald's The Princess and the Goblin, the primary antagonists of the story are subterranean goblins whose pet creatures are twisted versions of those above. One of which is "like a cat with legs as long as a horse's" and leaps from the night to attack the heroine princess. So, goblins who live underground and at least some of whose monstrous pets seem somewhat spiderlike in description. A stretch? Maybe. Maybe not.

Thirdly, there is E.R. Edison's The Worm Ouroboros. Another of Tolkien's self acknowledged influences. In this story, the antagonists are the men of the Kingdom of Witchland, who dwell in a dread swamp fortress called Carce, and whose mascot was the carving of a giant crab. Something sinister and spider like in that, methinks. It is not hard to imagine those stone carved legs beginning to wiggle in the fertile imagination of Tolkien.

I could go on to other instances of pre-Tolkien authors such as Edgar Rice Burroughs or H.P. Lovecraft who wrote of giant spiders of one sort or another. But did any of these influence Tolkien's Shelob directly? Who is to say? Then again, perhaps Tolkien himself said it best in his poem Mythopoeia:

Man, Sub-creator, the refracted light

through whom is splintered from a single White

to many hues, and endlessly combined

in living shapes that move from mind to mind.

I'm not a Tolkien expert per se, but I am a published poet in addition to studying history, meaning that I have a lot of literary training. My specialization for history/anthropology is the dharmic traditions, but in terms of literary studies I've extensively analyzed the works of early fantasy including Tolkien.

The shorter answer to your question is ... yes, I think. At least in the capacity of how that archetype is framed in fantasy (giant monstrous spider, all-devouring, representing primal force).

So the larger answer to your question is made complicated by the question's framing. Namely, by your focus on Shelob. See, Shelob was not the first archetypical giant spider which Tolkien wrote about. She was the first archetypical giant spider which Tolkien published writing about (Shelob is originally introduced in The Lord of the Rings). Tolkien actually wrote most of The Silmarillion before The Lord of the Rings, but he was unable to convince publishers to print such an unconventionally structured novel, hence him moving on to the more commercially viable LotR project. The Silmarillion contains a character called Ungoliant which, being another ravenous spider monster and the parent of Shelob, is arguably the prototype for the character. There are also giant spiders in The Hobbit, which predates The Silmarillion, but these don't necessarily conform to the standard fantasy archetype.

The Silmarillion is fundamentally a novel themed around creationism and gnosis. Let me burn really quickly through a summary of creationism in The Silmarillion. So the book starts with a creator-God named Eru creating a pantheon of deities called the Ainur. The Ainur are roughly analogous to the demiurge of Abrahamic theology. The Ainur split into two factions, one which remains in service to Eru (the Valar) and one which breaks off to follow the rebellious Ainur named Melkor (analogous to the "fallen angel" version of Satan). The Ainur then enter into the material world and engage in a vast struggle, with those in service to Eru attempting to ready the world for mortals to live there, and Melkor attempting to undo their work. Originally the Valar create two lamps to illuminate the world. Melkor casts these down and breaks them. The Valar then recreate light in the form of two great trees, a gold tree and a silver tree, which each give off a glow like the sun and the moon. They then create jewels called the Silmarils from the light of the two trees. However, Melkor allies with Ungoliant (a giant ever-ravenous spider) to undo this light. Melkor steals the Silmarils, while Ungoliant feasts upon the light of the two trees until they are depleted and destroyed. The Valar then create the moon and the sun in the image of the two trees. Melkor however still controls the Silmarils, which then leads into a long war in which the servants of the Valar (elves and men and dwarves) attempt to seize the jewels. Shenanigans ensue, thanks to the corruption inherent within the servants of the Valar, more specifically, the corruption of greed and the corruption of persuasion towards doubt. At the end of the war, the Valar decide that mortals are too easy to corrupt, so they separate their own realm away from the realm as inhabited by human beings, in a reshaping of the world which could be regarded as another instance of creation. The book ends with Melkor's second-in-command Sauron splitting his own soul in half to create the one ring (which would then go on to be the central storytelling device of The Lord of the Rings). Overall, The Silmarillion is about creation and recreation of divine light, with the transformation of creation being forever locked in conflict with the transformative force of corruption.

So where does this come from? Well, it comes from Abrahamic theology, but not necessarily the kind you might be expecting. See, when it comes to creation mythos, most people are used to the story of the creator deity making Adam and Eve, then casting them out of Paradise. But there's another version, one more commonly associated with Gnosticism as opposed to mainstream Christianity. The influence of this other version sometimes however gets recycled into traditions like medieval allegory, Christian mysticism, or also kabbalah. In the Gnostic version, the highest state of existence is Monad or the "unknown God". Equivalent to Monad is the Aeons, which represent the different aspects of God. However, God is both Monad and the Aeons, and Monad is just the many aspects of the Aeons represented simultaneously. In other words, Monad is God with indistinguishable attributes, whereas the Aeons are the attributes of God distinguished. They are linked through something called 'emanation', which is the state of being in which the perception of being is altered through being fractured. In gnosticism, emanation replaces the traditional notion of creationism (the latter of which Tolkien would have been more partial to). This link between Monad and Aeons is called Pleroma, and in Gnosticism, Pleroma is described as the origin of our world (ie both heaven and earth). In other words, we exist in the fracture of simultaneous existence into many distinguishable aspects of existence. In Gnosticism, the aeons are always balanced into male and female pairs, which is essential because gender represents a distinguishing characteristic, and the ultimate form of the aeons must amount to nothing distinguishing (being Monad). However, according to the Gnostic creation mythos, one Aeon named Sophia grew curious about the nature of emanation, and she attempted to create emanation of her own. She succeeds, creating the Demiurge, which itself is a representation of Sophia's desire to create. This is bad, because it throws off the balance of Pleroma. The Demiurge escapes into the material world, at which point it grows locked in an insatiably creating. The Demiurge itself then creates an order of lesser creators to assist it, called the archons. And here's where the real mind-fuck comes. The Demiurge is not aware that it originated as a subversive emanation of Sophia. As far as The Demiurge knows, it's just the highest order of creation, whose sole purpose is to create. So Gnostics believe that the thing that humans call God ... isn't actually God. It's the Demiurge. The Demiurge and the Archons are what we call God and his angels, and even they don't know what they actually are.

So to be clear, Tolkien was not a gnostic. But he was a British Catholic who also was very familiar with middle ages literature. At the same time, he was very much a skeptic of modernity. This often gets him described as "conservative", but as far as religion is concerned, it arguably made Tolkien more liberal. Again, I'm not saying that he would have upheld radical gnostic ideas. But he would have been aware that part of modernity was the church consolidating under more centralized control and centralized doctrine. The style of theology present within Tolkien's writing is more reminiscent of middle ages theology, the type which dominated before the church centralized and gained a stronger hand over this kind of stuff. Tolkien's theological portrayals are freer and looser. He experiments with different ideas within Abrahamic theology, and even with his own relationship to theology. Hence the themes of creationism. See, these themes extend beyond just the narrative. They enter even into the formal considerations of Tolkien's writing. The Silmarillion is deliberately written to resemble an old manuscript, the type which Tolkien often would work on as an academic translator. It's deliberately written to use a combination of styles, implying that different parts of the book came from different writers, and that at some point an editor compiled all of them. These processes are also forms of creation and recreation. Hence Tolkien constructs a metaphor between creationism and writing. He's portraying writing as the process of taking the light of creation and transforming its image, because the light of creation animates the world, and writers produce a transformed image of the world. Thus Tolkien views divine light as being that which is preserved through the transformative force of writing, and the temporal world as being those things which are possible to alter by portraying them in story. Hence he walks this unusual line between creationism and emanation, creator or the demiurge. To be clear, he leans towards the former. But he supplements it with ideas from the latter. The Valar literally operate in the role of the demiurge, with the maiar (their servants) being archons. The process of creation is portrayed less as divine plan and more of breaking and restructuring in a fashion reminiscent of gnosis.

Tolkien himself wrote of a giant spider decades earlier: Ungoliant, of whom Shelob was the last surviving offspring. There were also giant spiders in _The Hobbit, which predated Lord of the Rings, retconned as the offspring of Shelob.

Earlier than that, The Wonderful Wizard of Oz has the Cowardly Lion killing a spider the size of an elephant, which is how he became acknowledged as king of the beasts.

Those are the only ones I can think of, and since they're relatively recent it seems a little strange that there aren't others. But I don't think spiders have been generally conceived of as monstrous or frightening until fairly recently either. Historically, I believe they have been held up as examples of industriousness, not horror.