Well, some whales weren't hunted very often because they were too fast. Specifically blue whales were rarely taken until steam-powered whaling ships became available, because blue whales moved too fast for sail-driven ships to intercept unless they were exceptionally lucky. Humpback whales were also considered difficult because of their speed, although they were taken more often. Bigger whales are faster for physics reasons. (Look up "hull speed" for an explanation, if you're prepared for some calculus and engineering.)

To directly answer your question, there's two things you weren't aware of- first, whaleboats weren't trying to chase down whales from behind. (As the nautical saying goes, "A stern chase is a long chase.") A whaler would sail around to get in front of the pod of whales and only lower boats when they were in a favorable position to intercept the prey. Ship's logbooks frequently mention spotting whales but being unable to get to a favorable position to lower boats to attack them.

Second, whales, like most every other animal, don't spend much time at their top speed. Maintaining top speed is metabolically costly, so almost every creature uses a slower and more efficient pace unless there's a need for haste. Whales didn't seem to see human ships and whaleboats as threats to be fled from, so they'd just keep swimming along at a comfortable pace until it was too late, and one of them got harpooned.