1. Virgil generally uses the plural, Pergama.

2. The words aren't used completely interchangeably. Pergama refers primarily to the citadel of Troy and only pars pro toto to the entirety of the city. You can see the distinction made for example in III 349-51: procedo et parvam Troiam simulataque magnis Pergama et arentem Xanthi cognomine rivum agnosco.

3. Homer already uses (ἡ) Πέργαμος for the citadel of Troy (Ilias V 446), later Herodot (Histories V 43) and others use (τὸ) Πέργαμον (often also in the plural as (τὰ) Πέργαμα - poets like plural forms).

Why does he sometimes use Pergama when he could have said Troy? well, why does he use Ilium? Why does he have a dozen different ways of referring to Greeks and Trojans respectively?
variatio delectat and it's nice to have a selection of metrically different words that signify the same thing.