Questions about mansa Musa come up pretty frequently, so I'm gonna lift from earlier answers I have given to explain why the sources we have about emperor Musa's hajj in 1324 were written decades or centuries later, and probably exaggerate his hajj and his wealth.

Our earliest written source describing the hajj of Kankan Musa is by Shihab al-Umari who is relying on interviews from multiple sources to describe events he was not present for. He is writing sometime between 1342-1349, about events in 1324.

So, our earliest sources about these events is second-hand information, and al-Umari does not give numbers for how many retainers came with the mansa. For specific amounts, al-Umari only tells us that:

He had left his country with 100 loads of gold which he spent during his pilgrimage on the tribes who lay along his route from his country to Egypt, while he was in Egypt, and again from Egypt to the Noble Hijaz and back. As a consequence he needed to borrow money in Egypt and pledged his credit with the merchants at a very high rate of gain so that they made 700 dinars profit on 300.^1

Though elsewhere, al-Umari tells us of the repeated acts of generosity sultan Musa shows, giving out gifts to individuals in Cairo.

The other early source we have about this hajj comes from Ibn Battuta who visited Mali from 1352-54, almost 30 years after the hajj took place, and years after Musa died. Again, the information we are given comes from an anecdote by a person Ibn Battuta meets with.

We halted near this channel at a large village, which had as governor a negro, a pilgrim, and man of fine character named Farba Magha. He was one of the negroes who made the pilgrimage in the company of Sultan Mansa Musa. Farba Magha told me that when Mansa Musa came to this channel, he had with him a qadi, a white man. This qadi attempted to make away with four thousand mithqals and the sultan, on learning of it, was enraged at him and exiled him to the country of the heathen cannibals. He [the qadi] lived among them for four years, at the end of which the sultan sent him back to his own country. The reason why the heathens did not eat him was that he was white, for they say that the white is indigestible because he is not "ripe," whereas the black man is "ripe" in their opinion.^2

Again, very limited information that doesn't inform us about the full extent of the wealth carried or number of people on the hajj.

It seems that these figures of 12,000 slaves and 48,000 courtiers come from later recounting of the event, and are very likely to be embellishments. Therefore, I don't believe we can accurately say exactly how much wealth Mansa Musa carried with him on his hajj in 1324.

To take it a step further and stake a claim that he was "the richest man ever" based on an uncritical acceptance of those numbers isn't really supportable. We don't have enough information to make such a definite statement. Shihab al-Umari, Ibn Battuta and others all leave us with the impression that he and the Mali empire were rich, but don't tell us just how rich.

I've done a little more digging, and Delafosse's^3 french translation of the Tarikh al-Fattash quotes 8,000 men

....lorsque Kankan-Moussa partit d'ici pour se rendre en pèlerinage a La Mecque, il avait avec lui 8.000 hommes; l'askia Mohammed, que fit le pèlerinage plus tard, n'en avait qui le dixième, soit 800; enfin Ali ben Abdelqader, qui vint après eux, n'en eut plus que 80, le dixième de 800

But elsewhere gives numbers of 8,700 or 9,000 slaves.

Puis Farba, fondant en larmes et se frappant la pitrine, quitta le prince et se rendit a son campement. La, il convoqua les esclaves que, en un clin d'oeil, se presenterent devant lui. Ils etaint au nombre de 8.700 d'apres Baba, assara-moundio de la ville de Dienne; mai, suivant un autre, ils etaient exactement au nombre de 9.000.

[Though, these might simply be the slaves of Farba, and not represent the full retinue?]

But also mentioning 500 retainers accompanying the queen Inari Konte, wife of Kankan Musa.

On m'a raconte que Kankan-Moussa avait emmene avec lui son epouse nommee Inari Konte, accompagnee de cinq cents femmes et servantes.

Of course, the Tarikh al-Fattash is a historical chronicle written in the 17th century (most likely completed in 1664 by Ibn al-Mukhtar), and so was written about 350 years after Mansa Musa's hajj.

Some have cited "The Medieval Empire of Ghana" by J.H. Goodwin as a source for 60,000 slaves taken on the hajj, with concomitant amounts of gold. I've taken a look at the article, and it's not immediately clear where Goodwin derives his numbers from. Goodwin's references are all to secondary literature written between 1905 and 1948 (Goodwin's article is from 1957), and he doesn't provide a footnote explaining where the numbers come from. Goodwin does mention Ahmed Baba as a chronicler of Mansa Musa.

If Ahmed Baba is the origin of this number of 60,000 slaves and courtiers (I haven't found a translation of his work that has this claim), that is again problematic. Ahmed Baba al-Massufi al-Timbukti lived from 1556-1627, and his chronicles of Mansa Musa would also be over 250 years after the fact.

Also, as noted above, such a number would be in conflict with the Tarikh al-Fattash.

---

TL:DR- our earliest sources for mansa Musa's hajj are less extraordinary than the later retelling of the trip some 200 or 300 years later.

Personally, I am not very interested in trying to precisely calculate just how wealthy Musa was. Justifying whether he was "the richest man in history" is less interesting to me than the popular perception in the 14th-17th century Arab world and Europe that came to associate and enlarge the legend of Mali's wealth.

---

1 Corpus of Early Arabic Sources for West African History by Nehemia Levtzion and JFP Hopkins pp. 269-273.

2 Medieval Sourcebook: Ibn Battuta: travels in Asia and Africa 1325-1354

3 Tarikh el-Fettach ou Chronique du Chercheur by Mahmoud Kati ben el-Hadj el-Moutaoakkel Kati*, translated into french by O. Houdas and M. Delafosse. 1913

*in 1913 Houdas and Delafosse credited Mahmud Kati as the author, though work by Nehmia Levtzion and John O Hunwick since 2000 has now concluded that the Tarikh al-Fattash was most likely written by Mahmud Kati's grandson Ibn al-Mukhtar

Edit- Correcting name of Nehemia Levtzion

We have at least rough estimates on the wealth of far more ancient people, like Augustus Ceaser, but why is it so hard to pin down the wealth of Mansa Musa?

Do we actually have accurate ways to estimate the wealth of figures like Augustus?