It took hundreds of years for people in most parts of Latin Europe (France, Iberia, Italy) to stop considering themselves Roman. And in Greece, in the 20th century, there were still people around who considered themselves that.

"On 8 October 1912, during the First Balkan War, Lemnos became part of Greece. The Greek navy under Rear Admiral Pavlos Kountouriotis took it over without any casualties from the occupying Turkish Ottoman garrison, who were returned to Anatolia. Peter Charanis, born on the island in 1908 and later a professor of Byzantine history at Rutgers University recounts when the island was occupied and Greek soldiers were sent to the villages and stationed themselves in the public squares. Some of the children ran to see what Greek soldiers looked like. ‘‘What are you looking at?’’ one of them asked. ‘‘At Hellenes,’’ the children replied. ‘‘Are you not Hellenes yourselves?’’ a soldier retorted. ‘‘No, we are Romans."

Kaldellis, Anthony (2008). Hellenism in Byzantium: The Transformations of Greek Identity and the Reception of the Classical Tradition. Cambridge, UK: Cambridge University Press. ISBN 0521876885. pages 42-43

The short answer would be that Roman was not an ethnic identity to begin with, but a political and cultural identity that was defined by the relations between groups and elites from one hand, and the Roman state on the other hand, whose collapse removed a great part of what made one a Roman. It's arguably more complex than that, tough, as there's a notion of Barbarian identity being involved, which itself wasn't really ethnic too.

The collapse of the Empire in western provinces (which apart in Britain, Illyricum and partly in Northern Gaul; didn't meant the disappearance of the state itself but its translation into a Barbarian state) was an important social and cultural trauma for provincial elites, who since the IIIrd century underscored their social superiority trough costly display and honorability of functions : the source of their power, subsistence (wages and public service benefits) and social justification was out of their reach in the eastern provinces, and effectively gone.

To preserve their social standing, ambitions and expectations, Roman honestiores eventually entered into the service of the new Barbarian states who, after all, were essentially functioning as regional Roman administrations (although with significant regional variants from the quasi-imperial state of Ostrogoths to the light-weighted state of early Merovingians) and relied on a functioning public service to exist. But political lines had been blurred : while a Barbarian was defined by its following of a Barbarian king as part of a militarized people; and a Roman by its citizenship as part of the Roman political institutions; the distinction became much less clear.

While until the late Vth century, the tendency was rather Barbarians officers and generals adopting a public Roman apparatus and entering into the service of the Roman state (Merobaud, Stilico, Ricimer, etc.), this tendency shifted to the contrary : in order to ensure a position in the palatial network of Barbarian kings, Romans began to adopt the social codes of, importantly romanized, Barbarians such as names, fashion, functions. For example, the graves of Saint Lizier were made for people displaying all codes of what it meant to be a Frank in the VIth century (beer, weapons with runes and rings, horse burial, etc.) in a context of maintained romanity (wine, burial next to a Roman demesne and a chapel, the very fact they were buried and not cremated, etc.).

We're talking of a very gradual process of cultural and political mixing among elites : shifting identities from a Roman to a Barbarian one took decades at best (in northern Gaul especially, due to the collapse of imperial authority before Clovis' conquests, the conversion of Franks to a Nicean credo, and the absence of rules against intermarriage), but a couple of centuries elsewhere, a period of time on which people considering themselves as Romans or Barbarian lived together, defined not by their ethnic origin, but their adoption of particular social codes, a long coexistence that eventually made cultural distinction rather obsolete.

If you will, there was some divorce between displaying one's romanitas (roughly , romanness) trough cultural habits, display, functions or consumption which were shared with peoples calling themselves Barbarians; and a sense of more or less exclusive Roman identity which became rarer with time and essentially surviving in peripheral areas in the VIIIth century, namely southern Gaul and imperially-held territories in Italy.