I'm also curious how the idea of one China two systems was developed, and how seriously Hong Kong's political autonomy was taken during negotiations. Was it just the UK and PRC deciding things, or did they ask Hong Kong what it wanted?
In terms of negotiations, Thatcher did bring up the possibility of extending the lease on Hong Kong past 1997 in 1982 with Deng Xiao Ping during a visit to the PRC. However, this option was completely rebuffed. Deng cited the lack of true negotiation between the Qing and British governments when the treaty of Nanking was signed in 1842 and during subsequent diplomatic negotiations. The Chinese government made it it clear in the 1980s that any attempt to stop the PRC's "recovery" of Hong Kong would result in military action, with even Thatcher admitting that militarily the UK would not be able to hold the territory. The PRC government considered all of these treaties inherently "unequal" and thus believed it their moral imperative to fully recover from the losses of the Qing. The "unequal" nature of the treaties is somewhat debatable, emphasis on the somewhat. If you're interested in learning more about that I highly recommend Par Cassel's Grounds of Judgment: Extraterritoriality and Imperial Power in Nineteenth-Century China and Japan. He's a Chinese legal historian at the University of Michigan.
Sir Edward Youde, who was governor of Hong Kong from 1982 to 1986, proposed that he attend the UK-PRC negotiations to represent the interest of Hong Kongers was also met with anger on the Chinese side, largely because any Hong Kong participation could be construed as a recognition of the politically sovereignty of the territory. On an unrelated note, the name "Youde" is pronounced (without tones of course) as "Yoda" in Mandarin Chinese (but not in Cantonese), which I've always found entertaining. The Hong Kong Legislative Council also passed a measure declaring that any agreement on the handover be discussed in said council. This was received very negatively by the PRC government.
There was some discussion in the 70s on whether Britain could maintain administration of the territory "at China's discretion." This was labelled as the so-called "Macau solution." However, pointedly the British had rejected such an option after the 1967 Leftist Riots, when pro-communists in Hong Kong protested against British rule.
In short, there was no serious discussions within the UK government that the territory could be kept without a major military standoff and there was little willingness to start a war simply to allow the British to renege on the legally binding agreement that established the lease on Hong Kong. While there were some that undoubtedly want to retain control over Hong Kong, there simply wasn't a viable path to achieving that.
Edit: I'd also include Ian Scott's Political Change and the Crisis of Legitimacy in Hong Kong. It's from 1989, so it's a bit dated. However, it's a pretty good summation of the stances of the UK, PRC, and Hong Kong governments' stances during the 1980s negotiations. Of course this doesn't include much in the way of how Tian'anmen altered the way the handover was framed.
Edit: u/HKTenor has correctly pointed out that the lease was only for the "New Territories" and that Hong Kong island was fully ceded to the British in 1842. Here's the full text from the treaty on this subject:
"It being obviously necessary and desirable, that British Subjects should have some Port whereat they may careen and refit their Ships, when required, and keep Stores for that purpose, His Majesty the Emperor of China cedes to Her Majesty the Queen of Great Britain, etc., the Island of Hongkong, to be possessed in perpetuity by Her Britannic Majesty, Her Heirs and Successors, and to be governed by such Laws and Regulations as Her Majesty the Queen of Great Britain, etc., shall see fit to direct. "
The full text of the treaty can be found here.
Here's a map of Hong Kong that shows when different parts of the territory were acquired by the British.
The Chinese government had made it clear that it expected to fully recover from the unequal treaties it signed with the British and foreign powers. Most of the territorial concessions, most notably the French Concession and the International Settlement, were regained during WWII, but under the Japanese puppet government in China (Reorganized National Government of the Republic of China). The treaty for the recovery of the French Concession was actually signed with Vichy France. As I stated above, despite the wording of the treaty, Deng considered the ceding of Hong Kong an inherently illegitimate concession, and threatened military action if Britain refused to give up Hong Kong. While it sounds outlandish today, the British had already given up ever single colonial possession to this point, even if "legally" Britain owned said territory. In light of the general process of decolonization it makes sense that Hong Kong would be a part of this process in some way. Still, it wasn't considered possible (even by Hong Kongers) to fully achieve independence. The 1989 Tian'anmen Massacre instead led to increased efforts by local activists to speed up the process of democratization prior to the agreed upon handover of the territory to China in 1997.
Can someone clarify whether Hong Kong island and the new territories had the same status? I was under the impression that the UK "owned" the island in perpetuity but had a 99 year lease on the new territories.