As a Finn, my knowledge about early Finnish history is quite restricted. I know there was a religion involving similar gods to Norse mythology but with different weighings of the characters. The religion was in use with the Eastern Finns from Karelia, from whose stories Kalevala was composed, and included worship/respect of certain animals like bears. I also know the beliefs and traditions changed regionally.
But how I've understood it, all the knowledge from the general area of modern Finland we have now is through archaeological evidence, that there are no written records. Finns weren't vikings to my knowledge, but I've read that more or less recently viking swords have been found in the coastal areas. Did ancient forest Finns kill vikings, or were they purged? If there were any written records from the times before, did the christian Swedes destroy all of that "pagan" knowledge from the lesser Finns when they arrived? How much of a difference was there between for example a forest Finn from Northern Ostrobothnia area and from Karelia area?
These are broad questions, I will appreciate all answers focusing on any aspects about them.
First of all, the posts by /u/Platypuskeeper in What were the Finns doing during the Viking age? What kind of contact did they have with their Nordic neighbors?, Women’s status in Medieval Finland, and Slavic Vikings? will give quite good historical overviews of 'pre-crusading period' in now Finland to you.
While there seemed to be occasional skirmishes and raiding between the Scandinavians and Finns in the Viking Age Eastern Baltic (and well into the 12th century), they also had a more friendly interaction including trade on the former's 'Eastern Road (austrvegr in Old Norse)' into Russia. The Gulf of Finland and its surrounding regions like now SW Finland and Northern Estonia were, in a sense, the gateway to the Russian waterways to the wealth in the East, such as the Islamic silver.
As noted in OP, several Norse style artifacts, such as Viking style sword, have recently excavated roughly on this communication route, and some scholars argue that they were not solely monopolized by the Norse travelers, raiders, or some new settlers around the coastal areas. Some Iron Age native hill-forts around the gulf of Finland indeed also had these artifact, so some Balts and Finns in this region traded the Norse artifact with the Scandinavians, possibly together with their warrior culture. One of the few experts in this field of research, Mägi, argues that there was one shared warrior culture among the coastal people in Northern Baltic Rim (Mägi 2019: 46-52).
As for the 'Forest Finns' in inland area of now Finland/ Karelia, we have difficulty in presenting a trustworthy (coherent or diverse?) picture of their society. Even they were not isolated from the communication route between Scandinavia and now Russia, however. NW Russia and Northern Fennoscandia around the White Sea were famous as a product area of furs and pelts of exotic animals, and native hunter gathering peoples engaging trading of such animal products both with the Norse and Russian peoples latest since the (late?) 11th century.
From ca. 1100 onward, the citizens as well as a ruler of Novgorod, a commercial and political center of NW Russia, also began occasional raiding into Karelia (Eastern Finland).
If there were any written records from the times before, did the christian Swedes destroy all of that "pagan" knowledge from the lesser Finns when they arrived?
At least we are sure that the Finns didn't have any 'native' written media before the crusading period. Almost all the written evidence for them rely either on Scandinavian ones like runic inscriptions or some legends of raiding reflected in Old Norse-Icelandic historical writings or on Russian ones like birch back manuscripts excavated from Novgorod (or Novgorodian chronicles).
References:
I can only judge from the Russian perspective, but maybe that will be helpful.
First notes concerning Finns in Russian start to appear around the first half of 12th century. Majority of information about Finns from Russian chronicles (primarily those from the Republic of Novgorod) usually concern either trade or military activity with the latter predominating. Since the very beginning Russians seem to clearly distinguish southern Finnic people - Izhorians, Ests and Votes - who lived near the gulf of Finland and lake of Ladoga and are usually gathered in a single group called chud - and northern Finnic people from historic region of Tavastia (modern southern Finland) called Em or Sum in Russian (maybe you can recognize some Finnish words in these names?). Izhorians, Ests and Votes are sometimes described as allies and subordinates of the Republic while Em and others are stated to be main target for raids and plunder.
Decriptions of Finnish living conditions and behaviour are very scarce but it can be assumed that they were living in small, early feudal societies with some kind of warchiefs often engaging in trade and mutual raiding with Novgorodians. Republic considered southern lands populated by Finnish people as perfect spot for "soft" colonization with rules being so that tribes were to be allowed to live freely without major Novgorodian presence, although abiding to laws and demands dictated by the Republic and prodiving military support when asked. Later, after Swedes started fullscale colonization of the Finland, Novgorod considered local Finns already under its' jurisdiction as a valiable tool to oppose Catholic invasion - they would be baptized as Orthodox Christians and used as shield against Swedish incursion. Essentially both powers treated tribes of Finland to be very strong allies and good assets in waging war against each other.
After Swedes gained control of Tavastia, Finns appear to be usually described as forces utilized by Swedish armies in wars with Novgorod with later Russian notes sometimes calling them "pagans" or "witchers" for their seemingly incomplete christianization and strong ties with old believes.