Or was it just the case that it took 500 years for dying kings to not have a legitimate son?
In order to properly answer your question I will have to split the answer into two parts. In the first part I will aim to address the history of lines of succession in English history. Secondly I will try to describe the history of claims to the English throne by women.
Before the Third Succession Act of 1543 there wasn't any officially sanctioned line of succession for the throne. By line of succession I'm referring to a list of multiple individuals which clearly lists out the order in which they, and their children will inherit the throne. This Act named Prince Edward and any children he may have as heir to the throne, followed by Henry VIII 's other legitimate sons (if any were born, which they were not), then Mary then Elizabeth. The Act also allowed Henry to specify who would inherit next if all of his children were to die without having children of their own. Henry VIII named the children of his younger sister who was also named Mary as being next in line after his own children. Apart from Edward VI the only people who are placed in the line of succession in the 3rd Act are all women.
There were various decrees and acts relating to the succession before this since, The Third Succession Act as the name suggests was the third succession act passed by parliament. The First Succession Act of 1534 decreed that Mary I was illegitimate (a bastard) and not fit to inherit, and her younger sister Elizabeth would be Henrys' successor if he didn't have any sons. I'm not counting the 1st act as establishing a line of succession since there was only one person who could inherit due to that act while it was in effect. The Second Succession Act in 1536 declared Elizabeth to be illegitimate and barred her from inheriting the throne as well. This was all undone by the third act.
There was another other document about the succession during Marys' lifetime. The Devise for the Succession, which was the political will of Edward VI. The Devise unlike the succession acts were not passed by parliament, since parliament didn't have as much authority during the Tudor era as it would have from the 17th century onwards instead the Devise was signed by over a hundred nobles (most of whom later claimed that they were forced to sign it by the Duke of Northumberland).
In order to explain the Devise, I would first need to explain the reign of Edward VI, and the reformation. Edward VI inherited the throne of England after the death of his father in 1547 when Edward was 9. Therefore a regency council was set up for the duration of Edwards childhood to rule over England in his stead. Originally the regency council was led by Edwards' uncle Somerset (during this era English noblemen used their title as if it was a name). Somerset was removed from power by the council in 1549 and was eventually executed. After the fall of Somerset, John Dudley (who was the Earl of Warwick, but later became the Duke of Northumberland) become the head of the regency council.
During the 16th century Europe was undergoing a religious reformation, where Protestants opposed the religious leadership of the Pope in Rome. Edward and his regency council were Protestants however Mary was Catholic, and opposed the Reformation. Mary also happened to be a cousin of the Spanish King, and many people were worried that if Mary inherited the throne she would marry a member of the Spanish royal family and England would be ruled by Spain.
This was a common reason to oppose women succeeding the throne during the middle ages, since during that period many countries had laws specifying that a husband would have control over the property of his wife; meaning if they had a Queen ruling a foreigner could become the ruler of their country. There was a worry that if the Queen married the King of another country and had children their country would come under a personal union (multiple countries having the same monarch) there was a worry that the new monarch would rule from their fathers' country (although in Europe there were plenty of examples of someone ruling from their mothers country. For example Charles V, Frederick II, and Joan the Mad).
The Devise was mainly used to prevent Mary I from inheriting the throne, however it also removed Elizabeth from the line of succession. The original version of the Devise written in February 1553 didn't allow women to inherit the throne, instead if any of the women from the Third Succession Act (apart from Mary and Elizabeth) had a son, that son would inherit the throne. The second version of the Devise written in June 1553 changed just one line. Instead of saying "L Jane heires masles" it said " L Jane and her heires masles", this change meant that Lady Jane Grey became the heir to the throne.
Jane Grey was the daughter in law of Northumberland, it is presumed he arranged for her to succeed in order to cement his power (I won't elaborate on this since it's irrelevant to the question). Jane Grey became the first woman to be crowned Queen Regnant of England on 10 July 1553(however she isn't included in the official list of English monarchs). However Mary I and her followers didn't accept this and so they raised an army before Northumberland's' and his supporters could muster all of their forces; on 19 July 1553 the Privy Council(similar to the Cabinet) sided with Mary and overthrew Jane.
Previous kings of England before Henry VIII did die without sons. Of the 21 kings(not including Louis I, or Henry the Young King) of England between the Norman conquest and Henry VIII, 7 kings died without having legitimate sons or grandsons. Of these 4 were overthrown by their successor before/while they died(Richard II, Henry VI Edward V, Richard III), 2 of them had the throne pass to their brother without much difficulty(William II, Richard I). 1 of them is Henry I whose succession I will explain.
Henry I by the time of his death had no living legitimate sons, his only surviving child with his wife was his daughter Matilda. He had chosen his daughter to succeed him and made many prominent nobles swear an oath to support his daughter's claim to the throne. However after Henry died most nobles, especially in the South-east of England supported Stephen I, the nephew of Henry I. Stephen was anointed as King in 1135, and this led to an 18 year long civil war known as the Anarchy. Avoiding a conflict like this in the future is a part of the reason why Henry VIII was so desperate to have a son.
Sources:
The First Succession Act by The Parliament of England
The Second Succession Act by The Parliament of England
The Third Succession Act by The Parliament of England
The Devise for the Succession by King Edward VI
The Children of Henry VIII by Alison Weir
The Empress Matilda by Marjorie Chibnall