Short answer: succession law.

Longer answer: George I of Hanover, duke of Brunswick-Lüneburg, became King of England and Ireland in 1714 after Queen Anne. Anne had plenty of closer relatives, however all of those were catholic and according to the 1701 Act of Settlement Catholics couldn't inherit, taking the crown to George Louis Hanover, the closest protestant relative. His ascension to the English throne, put England-Ireland and Brunswick-Lüneburg into a personal union.

George II and George III also ruled over both the realms, but during the reign of the latter, Napoleon happened. Napoleon established the Kingdom of Westphalia which included Brunswick-Lüneburg. After the war, the Congress of Vienna re-established Brunswick-Lüneburg as the Kingdom of Hanover, and it went back to George III, lawful owner of the territory.

George the III, George IV and William IV consequently ruled over Hanover in a personal union.

In 1837 however, Victoria became the Queen which is where succession law kicked in. While England had Agnatic-Cognatic law, meaning that the crown would move to a female if there were no direct male heirs (probably a bit more complicated than that), Hanover respected Semi-Salic law which excluded accession to the throne by a female while any male of the dynasty survived. Thus instead of Victoria, her uncle in the male line of the House of Hanover, Ernest Augustus, eldest surviving son of George III, succeeded to the Hanoverian throne as Ernest Augustus I of Hanover.