My narrow specialty, medieval Scandinavia, had three (including the last during the Reformation, four) archbishops exiled out of the archbishopric.

• Archbishop Øystein Erlandsson (r. 1161-88): Exiled in England between 1180? and 1183, primarily due to his defeat in the Civil War in Norway. Stayed in the Abbey of St. Edmunds, by the discretion of King Henry II of England.
• Archbishop Eirik Ivarsson (r. 1188-1204?): sought asylum in Lund, Denmark (now Sweden, though) in 1990 where his colleague Archbishop Absalon of Lund had a seat of the archbishopric, after his conflict with King Sverre of Norway (r. 1177/ 1284-1202). He would reside in Denmark in exile until 1202, and given a small local church at Gumlösa in northern Skåne in the archbishopric of Lund, by discretion of Archbishop Absalon in 1192 (Ólafia Einarsdóttir 2002: 368).
• Archbishop Jon Raude (r. 1267-82): forced to be in exile in Skara, western Sweden between 1280 and 1282, due to his conflict with the secular aristocrats after the death of King Magnus the Law-mender (d. 1280) of Norway.

My answer below is mainly based on their experience, but I suppose that the easiest comparative example would be the Latin (Catholic) Patriarch of Jerusalem after the fall of the Holy Land in the Later Middle Ages, I suppose.

+++

Would he be forced to move his archdiocese?

Generally speaking, no (unless forced by superior authority - the pope).

He essentially becomes "titular" (arch) bishop, residing somewhere else in the original seat of the cathedral as assigned with the title. Latin Patriarch of Jerusalem, not resided in Jerusalem and instead resided in Cyprus "in exile" in the 13th and 14th centuries would be the best example. After the Reformation across Europe, some Scandinavian archbishops were also stay in Rome in exile, without losing their original title like "Archbishop of Uppsala".

To what extent the exiled archbishop keeps his original jurisdiction might not be so simple, but as for the temporary exile of medieval Norwegian archbishops, they still tended to be regarded as the head of suffragans in the church province. To give an example, Bishop-Elect Páll Jónsson of Skálholt, southern Iceland, took a visit in Eirik in Denmark in 1195 to get consecrated as stipulated in Canon Law.

It is worth noting that the archbishop in exile would not be the normal condition, so the church province (archbishopric) in question was often either under the interdiction or its secular ruler(s) was banned by the Pope, especially if the church province still keeps Catholic.

Who would come with him, in this circumstance?

It depends on case by case, but the exiled archbishop often took some important documents and liturgical manuscripts with them.

It is also worth noting that the archbishop usually has several suffragans (subordinate bishops) in his church province, and at least some of them often accompany (or forced to accompany) the archbishop in exile. As for the case of Norwegian Archbishop Eirik Ivarsson, he repeatedly urged his suffragans to estrange his rival, King Sverre, and the suffragans at first staying in Norway one by one complied this call and joined in Eirik in Denmark. On the other hand, in case of Archbishop Jon Raude, his most staunch supporter, Bishop Andres of Oslo, also followed him to Sweden.

References:

• Duggan, Anne J. "The English Exile of Archbishop Øystein of Nidaros." In: Exile in the Middle Ages, ed. Laura Napran & Elisabeth van Houts, pp. 109-30. Turnhout: Brepols, 2004. International Medieval Research 13.
• Ólafia Einarsdóttir. "Erik Ivarsson of Trondheim: Archbishop in Exile in Absalon's Lund 1190-1202." In: International Scandinavian and Medieval Studies in Memory of Gerd Wolfgang Weber, pp. 367-83. Trieste: Edizioni Parnaso, 2002.