It seems to me that it's Britain particularly that bears the brunt of criticism and blame for things like the slave trade and colonial brutality in general. It's like colonialism has been labeled as specifically a British thing and the colonial legacy of other countries has been forgotten.
The French, Spanish, Belgian, Dutch empires, among various others, were all heavily involved in the slave trade and wealth extraction in their colonies and all committed attrocities but I almost never hear those countries criticised to anywhere near the extent that Britain is.
I think it may be because the the usage of English and the lasting effect of British rule on its ex-colonies. The French and the Indochinese really messed things up and often committed far worse atrocities. However, the British empire had this sort of Halo effect on it and that caused more attention on it.