By the nineteenth century, naked power wasn't in vogue anymore, and the opinion of the common people mattered, so you're right to say they had to justify it. Really though, The White Man's Burden encapsulates the argument fairly well: the Africans were portrayed as backwards by people using pseudo-scientific racist rhetoric, and the conquest of their land was portrayed as a way to civilize them.

The labor they owed their new overlords was usually taken as compensation for the "education" they were to receive (though precious little was actually spent on education, and those who were educated were needed to man the bureaucracy on the cheap), though apparently some people claimed the labor itself was an act of education.

Throughout history, not just in the nineteenth century, it has been common to portray subject people as lazy (though, to be fair, they're probably not too keen to enrich someone else when they can grow just enough to feed themselves on their own plot of land). In Africa, this would then feed into the racist narrative that the natives would be going nowhere without the civilizing hands of Europe.

Oddly enough, despite all of the claims that it was being done to civilize them, western-educated Africans weren't held in any high regard either. They were regarded as mimicking rather than actually understanding, ensuring that it was pretty much impossible for anyone to ever make the argument "alright, we're civilized now" because the inferiority was regarded as genetic rather than educational.

If you want to read a book that's pretty down on empires, The Rule of Empires by Timothy Parsons has several case studies.