Did they believe them to be made of "stuff" which is unseen but physical?

Some Greek philosophers did! Though perhaps not in the way you're thinking of. The 'soul', in this case, is best defined as the animating principle of the body - the difference between a living body and a dead one was that the soul had departed (and so the word pneuma means breath, wind, and soul - a live body contains breath, a dead body no longer does). Animate beings contained souls, inanimate objects did not (and so Thales claimed that magnets had souls, because they moved and/or caused other objects to move, and therefore must have some animate motive force within them). This doesn't necessarily parallel the more, er, spiritual meaning of the term, and doesn't presuppose any sort of afterlife, continuing mind or personality, or potential for a disembodied existence.

(I am mostly ignoring the Homeric corpus here, singers of tales being unconcerned with ontology, and so not really caring what the soul was made of; the Underworld souls we see in the Odyssey are silent, mindless shadows, and only regain voice and life after drinking blood - which suggests they must be material in some way, or how could they drink? - but cannot be touched by mortals; Odysseus tries to hug his dead mother's shade, but she slips through his arms like smoke.)

Democritus, for instance, argued that the soul (like everything else) was made of atoms. Democritus argued that atoms, the smallest possible and indivisible basic unit of matter, differed from each other only in size, shape and texture; the atoms of the soul are very fine and spherical, and so are attached to each other only loosely. When their 'container' (the body) ceases to function, the atoms separate and scatter, to enter other bodies and form new souls. Heraclitus, on the other hand, seems to treat the soul as material and perhaps fiery ('it is death for the soul to become wet'), but also seems to claim that the soul maintains some form of existence after death.

Plato's dialogues seem to suggest that the idea of the soul as some sort of material thing was relatively widespread among Greeks (at least those familiar with philosophy). The Phaedo treats the idea of the soul being material as common sense:

“Men find it very hard to believe”, Cebes says at Phaedo 70a, “what you said about the soul. They think that after it has left the body it no longer exists anywhere, but that it is destroyed and dissolved on the day the man dies.” This view is restated by Simmias (at 77b) as the opinion of the majority (cf. 80d); note that the view includes the idea that the soul is a material thing, and is destroyed by being dispersed, “like breath or smoke” (70a).

(Socrates' opinion in that dialogue, on the other hand, is that the soul is imperceptible, immaterial, and immortal - for all material things are destroyed by time, and the soul, to be immortal, must be immaterial - and this Platonic ideal would go on to have tremendous influence on Christian conceptions of the soul.)

While Aristotle reasoned that the 'soul' was an emergent property of the body, the name we give to the proper, harmonious, functioning of all the components that make up living beings, later philosophers continued to see the soul as a separate, physical thing (Epicurus, like Democritus, was an atomist, while the Stoics saw the soul as a mix of the elements of fire and earth).

However, all that being said, this is about the spirits or souls attached to living animals/humans, and your question could be taken in a different way. For the animistic 'spirits' of nature, dryads and nymphs and such, or the less anthropomorphic numinousities and ancestral spirits of Roman tradition, you'll want a different rant.