Did the Aztec and Incan empires actually have that much gold, they never had a whole lot of metallurgical skills, but are still portrayed as having mountains made of shiney metals.
There is no reliable source on the precise amount of precious metals possessed by Andean natives at the time of Spanish arrival. But archaeological excavations in the Andes and South American coast have revealed a sizable amount of gold and silver objects, even after countless centuries of looting.
Some notable graves, including the Senor de Sipan of the Moche and the Tiwanaku Tesoro de San Sebastian, have been excavated intact and contain complete sets of gold regalia. These include elaborate chest-width pectorals, headpieces, and "garments" of hundreds of gold scales pieced together.
Andean gold jewelery features intricate detail, as seen on these Chimu ear spools. Note the fine incisions on the "bar." Moche gold beads could be as small as 2mm.
Gold versions of everyday objects have also been uncovered, such as gold kero ceremonial goblets and gold plated atl-atls.
We see gold usage throughout Andean history, back beyond even the Chavin culture of ~500 BC, the first pan-Andean movement. This particular example is actually gold-plated copper, a technological step beyond just plain gold.
So imagine you're a conquistador, and you find a fellow wearing a headdress of fine embroidery with this on the front, big 10cm disks like this on each ear, arm bands like this on upper and lower arms... even the pin that holds his cloak is gold. Oh, and this is across his chest. Of course you're gonna be amazed at the gold. We have historical drawings of each of these adornments, so it's no surprise that they could have lead to stories of massive amounts of gold among the natives. That's not to say there wasn't, as I just explained there was, but we have no reason to assume it was some mythical level of precious metals.
they never had a whole lot of metallurgical skills
While this is a common opinion, it's generally wrong. At big centers of production like Tiwanaku, innumerable metal processing tools can be found. Starting from at least the Middle Horizon (400 AD), with the Tiwanaku and Wari empires, people were crafting a variety of different alloys with metals sourced from across the Andes for use in specific contexts- architectural support, cutting tools, needles, jewelery, etc. Also keep in mind that gold is outrageously extractable and workable compared to, say, tin, which was used in many alloys.