I was reading Kalevala, the Finnish national epic, recently. At one point, a hero named Lemminkäinen is instructed to capture three animals owned by the god Hisi: a swan, a deer, and a flaming horse. Is this just odd translation, or did the Finns at this time have horses?
Most definitely they did. Although the oldest archaeological evidences (such as bits and other gear) of horses are dated around 300-500 BC, the Finnish words for those gears and for the horse itself (words like "hepo" and "varsa") are estimated to be a much older origin. But it was the time of Medieval and under the Swedish rule when finnhorse was breeded and people started to use horse to work in the fields insted of oxes.
Interesting side note is that riding a moose was forbidden during 1600s due to moose's cloven hoofs having a better carrying capacity in snow and swampy areas, thus giving Finnish criminals an edge compared to horse riding officials. I don't know what kind of horses were those but at least the finnhorse's height at the withers used to be around 120 cm at that time, compared to today's 155 cm.