The way I see it, either they mixed with the local populace, creating a genetic distinction between Sephardic and Ashkenazi, or they stayed within the Jewish population, resulting in a distinction between them and the local populace but not between one another. Another possibility is that they converted locals, but from my knowledge of Judaism, that was pretty uncommon (due to both a lack of tendency for the Jews to convert others, and a lack of tendency for others to want to convert), so I think we can safely rule that one out.
However, Ashkenazi Jews are distinguishable from both Sephardic Jews and Eastern European Gentiles (I don't know about the Sephardic side, but I assume they aren't genetically the same as Spaniards and Italians in the same way). So how did this happen? Because they either did or didn't mix with the locals, and this makes it seem like they somehow did both and neither.
I'm not a historian but I have a background in evolutionary genetics so I think I can provide some insight. The bulk of genetics studies has been done on ashkenazi jews and they show significant genetic relationship with both middle eastern populations in the Levant (ie Lebanon, south syria), as well as with europeans. This indicates both a middle eastern origin with substantial mixture with the host populations, which appears to be recapitulated in most or all major jewish communities. For a nice review on the subject see https://www.ncbi.nlm.nih.gov/pmc/articles/PMC3543766/
One may conclude that while conversion and intermarriage may be uncommon, it appears to have occurred frequently enought to see a substantial genetic signature.