It's kind of simple, really. If they're not spinning on their axis, oblong bullets tumble, and when oblong bullets tumble they will fly off in unpredictable directions. At a shooting range, this is called "keyholing", because of the slotted holes the bullets make in the target. Those keyholes are seldom hitting the bullseye, except by accident.

A round ball will also tumble if it's not spinning on its axis, but for the first 100 meters or so it will still go relatively straight- the effect ( a difference in air pressure on the sides of the ball) is not as great as for an oblong bullet. It will travel straighter over longer distances if it is spinning on its axis- that's why rifling works- but it needs much less spin than an oblong bullet- maybe one turn in 2 meters.

In the early rifles, it was necessary to wrap the ball in a patch in order to seal it and make it gripped by the rifling. Why patched oblong bullets weren't used is not really clear, but oblong bullets don't work as well with a patch- because they need a faster spin, the rifling has to have a faster twist and the patch can't really grip them tightly enough to impart the spin....or, more precisely, a patched oblong bullet that is able to be pushed down the barrel of a muzzleloader by a normal human with a typical ramrod will not be gripped firmly enough.

In the 1850's the French invented an oblong bullet that would expand to fit the rifling when the gun was fired, the famous Minié bullet. It didn't need to be patched. Soon after that, breechloading guns became standard, and so all bullets could be a very tight fit, and so all of them could be pointy.