So there's a couple factors at play here.

The first one is that there was quite a lot of migration between Jewish communities in Europe. While the more famous migrations actually wouldn't account for what you're describing--namely medieval migration of central European Jews to Eastern Europe (fleeing the crusades and being expelled) and early modern and modern migration from Eastern to Central Europe (fleeing the Khmielnitzky massacres and then later seeking economic opportunities), there was migration east late enough to produce what you're describing. One famous example is the Berlin family (Hebraized to Bar-Ilan by some branches), which includes a number of prominent Rabbis. The Berlin name came because the paternal ancestor had moved to Eastern Europe from...you guessed it, Berlin (though I'm not entirely clear on how far back).

Also worth noting that Jews of Eastern European ancestry may not be of entirely Eastern European ancestry. While Eastern Ashkenazi is the dominant ancestry of Ashkenazi Jews worldwide, the German Jewish population historically was decent sized (though many had moved back and forth across Europe over time), and a lot of Ashkenazi Jews have at least one discernible ancestor who was a German Jew in the fairly recent past.

But another, probably more important thing is that generally last names are not where a person lived at the time they recieved a last name, but the place they or their family lived before that. If everyone in, say, Albany took the last name "Albany", it would not be terribly useful to distinguish families or disambiguate family. Or would the fact that someone lives in Albany be of any interest to other people who also live there, to call them that. But if there's one family in Albany who moved from Syracuse, or a person who was born in Albany but left and is now returning from Syracuse, the fact that they're from Syracuse might be interesting enough to call them by that surname.

For example, Rabbi Jacob Emden ("the Yaavetz") lived most of his life in Altona. He lived for a very short period in Emden, but when he left "Jacob from Emden" became a meaningful way to tell him from all the other Jacobs in a way it wasn't when he lived there. Every Jacob in Altona could've just as easily been "Jacob Altona". His father, Tsvi Ashkenazi ("the Chakham Tsvi") got his last name because he was an Ashkenazi who left Ashkenaz--he spent much of his young adulthood in the Balkans, where his being an Ashkenazi was unique and worth giving him a surname for. If every Ashkenazi took the last name Ashkenazi it'd defeat the purpose of surnames in large part (also relevant that while "Ashkenaz" as a place is associated with Germany, Ashkenazim as a group include Eastern European Jews too). He kept the name even when he was back in Europe, even though it wasn't as useful in Germany as it had been in Turkey (though he did live in places with both Ashkenazi and Sefardi Jewish communities) This is why some people with the name "Ashkenazi" aren't Ashkenazi at all--an Ashkenazi ancestor moved somewhere Ashkenazim were interesting enough to be interesting, and took the name Ashkenazi even though back home that'd be uninteresting. Of course, a lot of people with the name are Ashkenazi--my guess is an ancestor moved to somewhere with non-Ashkenazim and then moved back and kept the name.