I'm asking this as a response (I suppose) to CGP Grey's video on how to become a British monarch. He pointed out some theoretical scenarios whereby the crown, left with no heirs, goes back up the family tree and follows down the next best preference for an heir, and repeats until it finds one that is not dead. What with the royal family going way back and shedding a lot of "less royal" branches that got scattered across the world, there's potential heirs all over the place (as noted in the video).
This, of course, requires an accurately recorded history of the royal family. How far back could this heir-finding stuff go and with how much confidence? Are there any odd holes or lost branches in the family tree (other than the secret bastard children royals are so fond of)? How rapidly does the list of potential heirs grow with each generation back? How many royals would have to die for the crown to have no heirs ("be lost"), if that is at all possible?
Typically, the crown does descend by male-preference primogeniture, as you describe in your first paragraph. ("Primogeniture" meaning the eldest son and his descendents inherit before the younger brother and his descendents; "male-preference" meaning every son gets picked before any daughter. So, Queen Victoria got the crown because she didn't have any brothers, but her second child Edward VII took the crown over his older sister Victoria.)
If this was the only rule, then as you say, we could go back as far as we have adequate records. However, Parliament has limited this by law. Most notably, the 1701 Act of Settlement, confirmed by the 1707 Act of Union, limits the Crown to "the most Excellent Princess Sophia, Electoress and Dutchess Dowager of Hanover, and the Heirs of her Body being Protestants." This followed upon the Glorious Revolution, where Parliament had invited in William III and Mary II to overthrow Mary's father James II, who was a Roman Catholic who'd been trying to promote Catholicism in England. Unfortunately, neither Mary nor her younger sister Anne had any surviving children, so Parliament passed the Act of Settlement to ensure the crown went to a Protestant branch of the family. This passed over James II's son "James the Old Pretender," who still considered himself king and would later lead the unsuccessful Jacobite rebellions. (One more restriction was later added: The Abdication Act 1936, upon Edward VIII's abdication, excluded Edward VIII and his descendents (of whom there were none).)
So, you cannot go back up the family tree any further than Electoress Sophia the mother of George I. Anyone beyond that is excluded by Act of Parliament. While this was highly significant during the early 1700's, George II had enough children that the limitation became of interest to academics and Jacobite rebels only. Currently, Queen Elizabeth is the tenth in line from Sophia. A 2001 study shows that Sophia had 4,973 living descendents; all of those who are not Roman Catholics (and haven't died since then) are currently eligible for the British throne. The lowest was a Ms. Karen Vogel of Germany, who told a reporter "It is really very comforting that one doesn't have to worry about" succeeding to the British crown.