Other than through military might backing an ambitious duke, or annexation by a neighbouring domain, was there a law of succession that applied when no legal heir could be found?
DIsclaimer: I know this to be accurate for England/UK, but I cannot guarantee that this will work exactly the same everywhere, but it is a broad question.
The general rule is that you continue to go backwards (and outwards) in the family tree until you find someone. With most royal families there will be extensive relations so it is essentially impossible to not find a new ruler through some distant connection. That said, it is entirely possible that one of the things you mentioned might happen.