There was very real European engagement with Africa in the early modern period: a massive slave trade, many trading ports, and huge Portuguese colonization. But why did colonization pick up so dramatically at the end of the 19th century?

There were a couple of reasons. Some historians have argued that it wasn't until the late 19th century that it was both profitable and technologically feasable to do so. The interior of Africa was, to borrow an expression from the time and the title of a famous history by Phillip Curtin on the subject, a "white man's grave." However, the advent of what Daniel Headrick called Tools of Empire, especially quinine to stave off malaria and steam engines to extract resources, made the interior colonization of Africa more feasible. In economic terms, the second industrial revolution made commodities like rubber much more valuable, and thus more profitable. Vast gold, diamond, and other precious metal deposits were also discovered--I'm sure you have heard of the Kimberly mine and the gold deposits around Johannesburg, which played no small part in Britain's expansion in the south of the continent. However, I think a lot of these deterministic explainations leave a great deal to be desired.

Instead, I think the international climate of inter-imperial competition has a great deal more to do with it. The completion of the canl at Suez suddenly made north and east africa of vital geopolitical importance to the British empire, as it secured the passage to India, the "jewel in the crown." Germany, freshly unified, saw the "realpolitik" of its cautious chancellor Bismarck go out the door and the ambitious "weltpolitik" of its Kaiser come to the forefront as Germany saught colonial holdings in Afrika. A newly unified Italy wanted the same. Republican France and Britain attempted to shore up their position, and imperialism became a zero sum game -- one needed to secure one's own holdings lest they fall into the spehre of another power. The Berlin conferences, which formalized the process by which a power could stake a claim over colonial territory, gave further incentives to powers to quickly carve up the continent and paint it various colors on their maps.

For a broad overview, check out Chamberlain's book.