This has to do with the Nation of Islam more than anything.

The mysterious Wallace Fard and Elijah Muhammed were leaders in the religious organization which considered themselves a true version of Islam with the main goal of improving life for African Americans.

There are also doctrines (Yacub) that prevent whites from joining and focus on empowering African Americans over a white people which are not in traditional Islam.

They had a surge of popularity with leaders like Malcolm X and member like Muhammed Ali in the 60s but splits in the leadership and subsequently the membership led to a whittling of numbers. Louis Farrakhan's branch seems to be slowly merging with Scientology, even.

Many members actually left the group to join traditional orthodox Islam as a result.

There are still a lot of references in popular culture to the NOI. In everything from Jay Z and Wu Tang Clan to Orange is the New Black and The Wire.

There's a really great personal account by a woman who grew up in the NOI in New York, I'll post it here as soon as I find it. EDIT: Little X: Growing up in the Nation of Islam by Sonsyrea Tate is a great introduction to NOI at the membership and cultural level.