Symbolism

In a time when cities had walls around them, the only way in or out was through the city gates. The gates were also the weakest point in a fortification for obvious reasons. The symbolism is that giving someone the key to the city means that the city trusts that person with the entire community's safety.

Origins

The ceremony has roots that date back to the medieval carnivals (think Mardi Gras) among the Dutch (Carnaval/vastenavond) and Germans (Kölner Karneval and Fastnacht) the week(s) before Lent. Traditionally, the mayor would step down for this period and power is transferred to the prince of the carnival (the coolest, party guy in town generally), who then returns the key at the end of Shrove Tuesday when everyone begins the fast for Lent. Even in this case, the handing over of the key was mostly symbolic as well. You can still see bits of this in Mardi Gras and Septuagesima today. The US, as a nation of immigrants, brought this tradition of handing over of the key as a mostly symbolic gesture to visitors and heroic residents.

I hope that gives you some insight into your question!

From New York City's website:

"The practice of presenting a key to an individual may be traced back to medieval times, when admission into a city was hampered by many legal restrictions, as well as by walls and locked gates. The key symbolized free entry."

In ancient times, most European towns had walls and gates to protect from outside attack. Visiting dignitaries were presented with a key to the city gate to symbolize trust; those who had the key could come and go as they pleased with the assumption that they were friends of the city. Nowadays it's more of a gesture; symbolic and nothing more since you can move amongst cities as you please in most cases.