Yes, George VI was quite reluctant to take on the throne. There is a quote from George VI's diary that upon discussing Edward's abdication with his mother Queen Mary, he broke down into tears.

As you saw in both The King's Speech and the Crown, George VI ("Bertie") had a substantial stammer. As a child, he was also said to be knock-kneed, awkward, and shy. Compound these issues with a stammer and a fairly critical father (George V), he grew up as a timid, insecure man who shied away from the public eye until he was forced to take the crown by the abdication of his elder brother.

In addition, the time period in which George VI took the throne was not terribly stable. His brother's abdication rocked the monarchy, and deciding what to do with a former (disgraced?) king was challenging. Bertie ascended the throne in 1936 and there was already rumblings of discontent on the continent. It was a daunting situation to enter for anyone, let alone someone who was far happier living a "simpler" life with his wife and children.

An interesting read on George VI is this book by Sarah Bradford. I found it a little critical of Edward VIII, but to be fair, George VI and his family did not view Edward very favorably after he abdicated.