It kind of depends on when you're talking about and who's doing the signifying. In the later Middle Ages, and particularly among groups outside of Europe (Byzantines and in the Islamic world), what you say is pretty much correct. All westerners were "Franks," regardless of any political or ethnic affiliation. "Latins" was a co-terminus identifier. Within Europe though, they didn't lump quite like that after ca. 1100 CE and so would identify people as "Christians" (as opposed to "heretics," "Jews," or "Muslims") or by their political affiliation ("French," "German," or more specifically "Aquitanian," "Breton," "Swabian," "Catalan," etc.).

But in the early Middle Ages, it's more complicated. "Latins" wasn't really used but "Franks" was a political, religious, and ideological identifier. A "people" (gens) was a cultural affiliation that was porous and changeable. People could move between them by being conquered, by conquering, by intermarrying, etc. What Charlemagne and particularly his successors did was to open up Franci so that the peoples they conquered could be subsumed into the dominant group -- into the new chosen people, who proved they held God's favor by their constant victories in the 8th century. But once that ideology's established, it's really hard to undo. If things are going badly, it doesn't mean you're not the chosen people anymore, it means that you've offended God and need to get back into His good graces by acting rightly in the world. People still want to be "Franks," even through the 10th and 11th centuries as it becomes a superlative. Just to give 1 example, Notker the Stammerer at the end of the 9th century, as the empire's falling apart, could still write:

Now, when I said Frankland..., I meant all the provinces north of the Alps;...so at that time, by reason of the glory of Charles, Gauls, Aquitanians, Æduans, Spaniards, Germans, and Bavarians thought that no small honour was paid to them, if they were thought worthy to be called the servants of the Franks.

In other words, in that period, you too could become a "Frank" by acting rightly in the world -- by adhering to proper religion, by conquering your enemies, etc. Read more here.