The main reason he never visited Hanover is that it was a possession of various foreign armies until 1814, and did not become an independent kingdom until then. George III suffered from severe mental illness (historians don't agree on what it was, although porphyria and bipolar disorder have been mooted) throughout his life, but this became severely acute from 1810 onwards, and it was not at all clear that he was lucid enough to even understand that he was now king of Hanover, let alone being able to travel.

I wrote more about the Hanoverian situation vis-a-vis Great Britain here. I'll copy-paste it below:

I think you may be misunderstanding the legal situation just a bit -- several kings and one queen of Britain were members of the House of Hanover, but Britain was not "controlling" Hanover. Rather, the kings of Britain were electors and dukes of Brunswick-Lüneburg (generally/informally referred to as electors of Hanover) and were kings of Hanover after 1814 through 1837, when Victoria was barred from the succession under semi-Salic law.

(Technically, George I through III were kings of Great Britain and of Ireland, and George III-IV and William IV were monarchs of the United Kingdom of Great Britain and Ireland. If you want to get technical.)

Anyhow, the electors of Hanover just happened to occupy the same body as the kings of Britain; it was a situation that is analogous to the situation that James VI/James I inherited as dual monarch of Scotland and England.

From what I know about the situation, Hanover continued to have local laws apply in its territories, but being a possession of the kings of Britain became a battleground in the Seven Years War and in the Napoleonic wars.

Early in the Seven Years War, Hanover was overrun and was forced into a state of neutrality with French occupiers; George II eventually revoked that treaty and, with the help of a Hanoverian-British army and allies from several German states, retook the territory from the French. After 1758, Hanover was not affected directly by the war.

In 1801, though, Brandenburg-Prussia invaded Hanover with Napoleon's urging, although troops withdrew after the Battle of Copenhagen when the Brandenburg-Prussia/French alliance fell apart. Hanover was then invaded again in 1803, after Napoleon broke the Peace of Amiens, and went through several occupations by foreign armies until 1814 (the sequence is long and confusing even by the standards of the Napoleonic wars). The Hanoverian army was dissolved, but many of its officers and men traveled to Britain and formed the King's German Legion, which fought throughout the Napoleonic wars until the fall of France in 1814 and at Waterloo in 1815. Interestingly, during the period of occupation, the Hanoverian ministry continued to operate from London as its own diplomatic service.

After 1814, Hanover became a kingdom, rather than an electorate (the Holy Roman Empire had been dissolved in 1806). The monarchs of the House of Hanover became kings of Hanover through 1837, as I mentioned above.

Anyhow, hopefully that will help. It's a complicated history for sure.