The earliest seaworthy vessels were simple canoes made of bark or other simplistic carved logs. The earliest known excavated canoe dates to 8000 BCE and seems to have been carved with antlers or flint. The history of boating certainly starts waaaaay earlier, as indicated by the settlement of Australia and places like Sardinia and Corsica, but the Pesse Canoe is almost certainly emblematic of the boats that humans would have used in the roughly 50 000 years preceding it. The scope of 'prehistoric' is a bit large, but certainly by the beginning of history (say, 3000 BCE), boating had dramatically evolved from the humble beginnings of the Pesse canoe. As you can note from your link, the rowing canoe had been invented by 5000 BCE, and by 3000 BCE the Lapita culture had successfully navigated to the islands of the pacific using - you guessed it - canoes, sidelashed to each other to carry goods between the canoes, powered by sails and oars.

I think it's important to acknowledge that the majority of prehistoric sailing was not sophisticated marine navigation but (relatively) very short skips across bodies of water, or catching currents downstream. Canoes were, and continue to be, most popular on rivers. Canoeing on rivers doesn't require oars since small rivers can carry matter downstream without human intervention, and early humans would have used their primitively carved canoes to harness this impetus.

Apart from that, consider an experimental archaeological approach - although oars with carved paddles were a ways out, you know what isn't hard to make? A stick. You know what else is vaguely paddle-shaped? Hands. Across short distances (even long by our perspective - the English channel is certainly short enough for this) you could paddle a simple canoe with your hands, kneeling on the bed and putting your hands in the water on either side to push yourself along.

For upstream / shallow stream navigation poling is an effective tool that requires more-or-less no technological innovation, only technique. Find a suitably long stick and push yourself upstream.

What did prehistorical humans use boating for? Well, this is beginning to get into archaeology rather than history, but I think it's safe to say that faster traveling speeds and the ability to haul material up and down rivers was conducive to productive hunting and basic trade. Early humans would have certainly exploited the value of being able to cross bodies of water quickly to chase food, trade with other settlements, and colonize new areas without the arduous need to swim (remember that crossing the Channel by swimming is still quite a feat - try doing that with a whole colony of women, children, and elders). Polynesians famously used personalized canoes to attack enemies on nearby islands in Hawai'i, with each warrior rowing themselves to and from, and that's open-water rowing. So it's application wasn't purely peaceful.

I believe your confusion is due to terminology.

Paddles came before oars. Paddles are a simple piece of wood that the paddler sticks in the water and pulls. Think canoeing.

Oars are connected to the boat with a flexible joint and utilize mechanical leverage. The rower can lift the end of the oar out of the water, pivot on the joint and place the oar back in the water and use body weight in combination with muscle effort to propel the boat forward.

u/Patchknight has already discussed early boats, so I'll just add:

how were early oarless boats used

Usually with paddles. The date for the invention of oars that you refer is specifically for oars, not paddles. These are considered different things, because they're used differently. Oars are pivoted on the side of the boat between the handle and the end that goes in the water. To pull the oar backwards in the water, the handle is pushed/pulled forwards.

In contrast, a paddle is held in both hands, and isn't pivoted on the hull. For a single-bladed paddle, to pull the paddle through the water, the paddler pulls the paddle back with both hands.

The timeline entry specifically notes "rowing oars" to distinguish between such and steering oars and sculling oars.

As u/Patchknight notes, poling was also used. Hand-paddling was used, and also towing/pushing by a swimmer.