I just found out it was Mediolanum (Ancient Site of Milan), and later Ravenna.
Rome remained the capital, inasmuch as the Senate continued to reside there. But the emperors by necessity needed to move around the empire, and there were a number of cities that functioned as imperial residences and centers of imperial administration when required—besides Ravenna and Milan, there were also Arles, Trier, Thessalonica, and Antioch, among others.
The emperors never abandoned Rome altogether—several of the last emperors resided at Rome for part or all of their reigns, and in civil wars controlling Rome was critical for establishing legitimacy. But there were several reasons Milan was often a more practical residence: it was closer (by land) to the rest of the empire, saving the emperor and field army weeks of travel time in an emergency; it was more politically independent of the Senate (there was a lot of friction between the emperors and the Senate in the period when the former were Christian while the latter was still predominantly pagan), and it was safer from a potential sea attack from Africa (which was periodically in rebellion, and later occupied by the Vandals). And Ravenna was safer from a land invasion, and could be supplied by sea from the east.