There are many different types of illumination, depending on where you are and what resources are/were available.

Torches and reed lights were quite popular. And there are even combo rushlights/candle holders that were used during the European Middle Ages.

Oil lamps were also very common. I know you asked about pre-oil, but I suspect you mean glass lanterns. Clay oil lamps were the usual method of lighting from at least the Bronze Age in the Middle East. We see lovely clay oil lamps from Classical Greece and Rome as well. These would have burnt olive oil, as well as any other oil they might have had.

Similarly, Inuit people used similar oil lamps, burning whale or seal blubber/oil.