The only real source I have is Wikipedia, but ritual type-activities such as burial and other things that might suggest some form of spirituality/religion are literally about as old as the human race (Neanderthals did such things too, IIRC they have found Neanderthal graves with flowers and other grave goods left in them).

For something a bit more specific, I guess Venus figurines? If you don't know what those are, it's a collective name for a huge number of Paleolithic statuettes of a stylized pregnant woman, an early fertility goddess, found across Europe.

A statue of a snake seems to be the oldest proven find to date.

A new archaeological find in Botswana shows that our ancestors in Africa engaged in ritual practice 70,000 years ago — 30,000 years earlier than the oldest finds in Europe.

You might be better off asking /r/AskAnthropology for this. We have no historical records of the first instances of spirituality. Two other users have posted about artifacts and burials, but we know very little about what they were used for.

FWIW, before the beginning of agriculture all humans lived as foragers ("hunter-gatherers"). There's no common thread of spiritual thought among foragers and it's usually inappropriate to refer to their spirituality as "worship"- the San, for instance, believe in a god and a "devil", but they don't do anything that can appropriately be called worship.

While there's no mode of religiosity uniting foragers, many forager societies have some kind of shamanism (loosely defined- some anthropologists prefer a much more limited definition); shamans are people who go into an altered state of consciousness to reach a spirit world and perform tasks to benefit people- collect supernatural medicine, converse with the dead, or enlist the help of supernatural beings. The actual manifestations of shamanism can be very different from each other. The "classic" shaman is a special person set apart from society who might perform supernatural tasks for favors; in contrast, among the Kalahari San nearly half of all people are shamans and they enter into an alternate state of consciousness during a communal dance. Shamanism is likely very old, but we can't know how old and we can't know what (if anything) preceded it.

Not being knowledgeable in historical religions I can't vouch for the influence or role of shamanism in them.