The legends of this "silent trade" in writing go back to Herodotus (!), who writes:

Another story is told by the Carthaginians. There is a place in Libya, they say, where men live beyond the Pillars of Heracles; they come here and unload their cargo; then, having laid it in order along the beach, they go aboard their ships and light a smoking fire. The people of the country see the smoke, and, coming to the sea, they lay down gold to pay for the cargo, and withdraw from the wares. Then the Carthaginians disembark and examine the gold; if it seems to them a fair price for their cargo, they take it and go away; but if not, they go back aboard and wait, and the people come back and add more gold until the sailors are satisfied. In this transaction, it is said, neither party defrauds the other: the Carthaginians do not touch the gold until it equals the value of their cargo, nor do the people touch the cargo until the sailors have taken the gold. (Histories 4.196)

You can read this passage a couple ways. One, Libya is the standard term for Africa and the Pillars of Herakles is the Strait of Gibraltar, so Herodotus is relaying the actual method of trade between Carthage in North Africa and some civilization perhaps on the northwest coast. Two, ancient Carthage had the reputation of shrouding its trade partnerships and commercial activity in pitch-black secrecy, especially the source of some of their trade goods outside the Mediterranean ring, so fanciful legends develop around it.

In the Middle Ages, the trans-Saharan gold trade gives rise to similar secrecy and a similar desire to know where that gold is coming from. (I mean, if you were the Ghana Empire, wouldn't you want to protect the location of your gold mine from northern traders, so you could continue to profit as miners and mediators?) Perhaps inspired by Herodotus, several Muslim scholars pick up the silent trade legend.

The earliest reference to drums, specifically, that I'm aware of is Yaqut al-Rumi in the 13th century ("Yaqut the Greek," actually born in Constantinople!), who writes:

They stop here [in Ghana] and recover...and recruit locals for their negotiations with the owners of the gold. They proceed until they arrive at the place that divides them from the owners of the gold. There they beat the big drums that they have brought...When the traders know that the owners of the gold have heard the drums they produce whatever they have brought of the goods. Then they withdraw to a distance equivalent to one traveling stage. Then the others come out and put certain amounts of gold beside each ware, and leave. The traders come again and each of them takes the gold found beside his merchandise, which he leaves there. Then they beat their drums and depart.

But already Yaqut's account doesn't make sense: the outsiders recruit middlemen to negotiate, and then engage in a method of trade that very explicitly cuts out middlemen.

In fact, among the Arabic sources for the west African gold trade in the Middle Ages, the writers who actually visited West Africa like ibn Battuta describe much more standard trade practices of meeting merchants in cities to conduct the exchange. Yes, the gold suppliers were very keen to keep their sources a secret (later empires controlled three gold mines, not just one). But the people who witnessed the exchanges firsthand did not observe the silent trade.

It's a tantalizing legend, of course. Europeans in the late Middle Ages and early modern era picked it up, and even applied it to other areas of Africa (Axum/classical Ethiopia, for example). You can even find scholars repeating it as truth today. The explanation usually runs that silent trade allows people who don't speak the same language to participate in exchange, and allows the suppliers of the tin, gold, salt, etc. to keep the origin of their goods a secret. But you can see how that's an ex post facto explanation that doesn't really hold up (above all to Occam's Razor, but also to the rest of the historical evidence).

Mauny, Sundstrom, and Farias have probably been the most important scholars in critiquing widespread modern acceptance of the "silent trade."

Meanwhile, is Scales of Gold any good? I didn't love the Baroque Cycle (too bad; Stephenson is one of my favorite authors when he sticks to sci-fi), but I've had my eye on the other series for awhile now.