The Sack of 410 is more a political-military slip-up than something really planned by Goths : before 408, Alaric attempted to negotiate with Ravenna in a strong position to obtain lands in Pannonia and Noricum, subsides, political acknowledgement and favourable peace; so far, Stilicho's efforts managed to push-back a lot of Gothic advancements in Italy but after his execution, Ravenna couldn't oppose Goths efficiently and the anti-Barbarian faction mostly antagonized Alaric (in particular with the violent purge of Barbarians in Italy) without clear political or strategical benefits.Then Alaric elected to invade Italy with his army, more to pressure the imperial court than to really go into a rampage, and encircled Rome that was protected by a restored wall since 402 thus except some skirmishes outside the city, it was mostly about depriving Rome from supplies, Ravenna being unwilling to negotiate and unable to intervene.

Thus,after a rather pitiful demonstration of strength, the city of Rome accepted to pay a tribute to Alaric,mostly taken from senatorial fortunes and pagan temples offering and decorations. It was humiliating, but overall, it represented the best alternative giving the situation.It is quite possible that it's at this point that most of what could be looted at the Pantheon (statues, offerings, bronze decorations, etc.) were taken away.

This forced Honorius to compromise and the siege was lifted as an agreement was close to be made, with Rome's elite transmitting Alaric's terms. Except, it wasn't and Ravenna went back to a fierce anti-Barbarian stance without having the means of such politics. Alaric tried to moderate, in a Roman fashion, his demands but to no avail.

Henceforth, pulls an imperial challenger with the support of Rome and attempts to form an alternative to Honorius to obtain what he wants;and move to Ravenna with Attalus who refuses to move in Italy to obtain necessary supplies for Rome; Alaric being in the misdt of a Romano-Roman dispute on this regard; and ends up breaking with Attalus as the alliance went nowhere, negotiations with Honorius break in shady circumstances, and Goths are back to besiege Rome in 410,hoping to use it as a leverage against Ravenna's obstinate attitude.

It's not clear at all how Goths managed to enter Rome, probably trough the Salarian Gate : damages are obvious in the city, which can be attributed to sheer plunder especially in aristocratic houses, political point on public space or some aristocratic houses, and crushing an hastily gathered defence. Overall, the damages are mostly located in the southern part of the city, on Aventinus and Caelius, and seem after a moment to be the doing of scattered attacks and acts of plunder, rape and killings, than an actual plan.

Alaric, eventually, intervened to make some damage control : his plan was to pressure Roman and to make them accept his terms, and receiving something for the waste of time would it to pay his troops; not a wanton destruction of the city for the sake of it that would have obtained Goths nothing more than a tribute would have. Besides these material considerations, Alaric wanted to display himself as both a Barbarian king and a Roman general, and contemporary authors often stress his Roman-like virtues.

Assuming that Goths in a looting rampage would have gone out of their way to destroy the Pantheon (not just damage it as it happened with some churches in spite of Alaric's order) but to destroy what was not yet a church at this point, and with little to loot instead of partaking in the aforementioned acts and abuses of the population; it's not a given Alaric would have simply accepted such an utter and useless destruction.

Attila himself, while having the numbers for that, elected to obtain a tribute rather than sacking Rome, maybe with the remembrance of the swift end of Alaric after his troops plundered the city and the important problem it represented in Gothic negotiations with Romans.

The sack of 455 happened in a different context, arguably : Genseric, frustrated of a possible dominant position in Italy after the death of Valentinius III, of a political acknowledgement of his dominance in Africa trough an union between Eudoxia and his son Huneric, raided Rome to make a political point, legitimizing it by the call for help sent by the imperial daughters.

The city is basically lost even before Vandals land in Italy, with people and critically the armies fleeing with all they can; the remaining notable families welcoming him as the restorer of legality; which does not prevent the bulk of the population fearing another 410. The pope Leo then breaks an agreement : Genseric and Vandals will be able to take a tribute from the city as they see fit and to organize themselves to take it in Rome, hill by hill, neighbourhood by neighbourhood. Basically, a legal sack that lasted two weeks, much more thorough than what Goths did.This time, the integrality of the imperial treasure is taken, churches aren't spared, etc. to the point at least one ship sunk under the weight of the loot.Genseric doesn't want to destroy Rome, he wants to empty it of its wealth, to take slaves but as well decoration as statues are taken to embellish his African capital : there are traces of destruction, possibly politically motivated as the burning of the imperial palace, probably more or less spontaneous in the southern part of Rome in the same conditions than in 410.

The third sack of Rome, by Ricimer's troops in 472, isn't well attested archeologically : as usual, spontaneous destruction and damages due to the first entry in the city as it was extremely common for any successful siege in Antiquity, and looked more as a purge against Anthemius' supporters (or alleged supporters) and a military plundering of their belongings; while Barbarians and Roman troops celebrated their victory with the usual mix of killing, looting and raping. There too, the destruction of the Pantheon would have been pointless, especially for a Ricimer willing to preside over a "reunified" Roman elite in Italy to the point respecting last rites to Anthemius' corpse.

Eventually, nobody had a real interest destroying this building, going out of their way in their looting of aristocratic housings or even churches; just for the sake of destroying it : the image of Barbarians destroying everything in their way by sheer "vandalism" is rather a stereotype, reinforced by later events and especially the Sack of Rome of 1527 by Protestant Germans troops, likened to Goths and Vandals in contemporary literature

- Roma capta, Il sacco della città dei Galli ai Lanzichenecchi; Umberto Roberto; 2012 (french edition, translated by Yann Riviere).