To the extent that many African countries have low GDP, colonialism often played a role.
More specifically, it was not just colonialism, but extractive colonialism. Some colonies, like New Zealand and the United States, were settlement colonies that developed representative institutions, taxation capacity, and strong governments which were able to offer protection against violence (e.g. police, military). Upon independence, those former colonies were able to utilize the developed institutions to establish strong and independent states.
Extractive colonies were those where the colonizers set up institutions only to the extent needed to extract and ship resources back to their home countries, which is to say, not to a very large extent.
In extractive colonies, the colonizers only needed to use force to keep slave labour in line. The government would generally not be run by the indigenous population, but rather a colonial government composed of people sent to govern that territory specifically.
Generally speaking, those states did not do too well. However, colonization doesn't always fully explain any given country.^1
Consider Nigeria, for example. When Nigeria achieved independence, people had high hopes that democracy would do well, as at least in the south, the British had set up institutions staffed by the indiginous population, many of whom had been educated. However, things went downhill after Nigeria discovered oil.
This is known as a resource curse: states which rely on a single resource don't have the incentive to develop a strong taxation capacity, and without a strong taxation capacity they generally don't develop a strong relationship with the population and don't deliver the services expected of a government. Instead, in Nigeria and elsewhere, foreign capital flows directly to the government, which then consolidates power in a system where politicians are unaccountable to the public but incentivized to hold on to power in order to line their own wallets. We can see this at play throughout the Middle East, Russia to some extent, and Venezuela as well.
Thus, Nigeria's future was mired by civil war, military dictatorships, and flirtation with democracy, but today is still very much a weak state.^2
In weak states, economies tend to be weak as well because, for example, there isn't a strong enough government to enforce property rights, and to keep people safe. In addition, other institutions such as universities don't develop to the same extent that they do in territories with strong states.
In sum, colonization plays a role, but there are other relevant factors as well. Ultimately, state weakness is mechanism through which a colonial legacy leads to modern problems.
EDIT: Some clarifications and a link