So obviously Robert Peel and the invention of a modern police service was the century after the rebellion of the colonies and establishment of the USA, but why is there such a dramatic difference in terms of policing methods, interactions, ethos, etc?
Why are police in the Commonwealth seen as civilians, who can't order people about for no reason, whereas in the US they issue "orders" to ordinary citizens, and expect to be obeyed?
I understand the rationale/arguments made for carrying guns and using a lot more force (i.e. armed populace, back up being so far away, etc.) and I get them. But what's the historical reason? How did the development of a police force in the US differ from Robert Peel's noble institution? Was there a discussion had, in the US, about the formation of police forces?
I can answer from a legal perspective and the history of law as it pertains to police forces, and the impact of those laws, but there are still a wide variety of theories as to why the specific attitude about police has become what it is in America.
From a legal perspective, the police force as we know it in America began in 1967. President Lyndon B. Johnson commissioned a study into crime and the criminal justice system, aptly named The President's Commission on Law Enforcement and Administration of Justice. The report was finalized in the same year and was titled The Challenge of Crime in a Free Society (it can be found in full here https://www.ncjrs.gov/pdffiles1/nij/42.pdf).
The report detailed almost every aspect of criminal behavior as well as the three individual branches of law enforcement (courts, corrections, and the police force). After it's review, it provided a lengthy list of recommendations and strategies to combat crime for local governments, state governments, the federal government, and for citizens and nongovernmental organizations. I won't detail those recommendations here, but the report I linked above does provide a good table and summary.
To get back to your question, after the report was released, we began to see the shift in attitudes towards police. Prior to the report or the Commission, United States police forces were very much in line with the English common law system (a mix of citizen volunteers and a system known as posse comitatus). In 1953 Earl Warren replaced Fred Vinson as Chief Justice of the Supreme Court of the United States. The Warren Court over the next 16 years would expand civil rights, civil liberties, judicial power, and the power of the federal government. Among the major decisions of the Warren Court were Mapp v Ohio in 1961 and Miranda v Arizona in 1966. These two cases set national standards for policing, specifically as they relate to the 4th and 5th amendment rights of criminals. In 1967, for the first time, there were now national standards for policing as well as comprehensive details about crime and how to solve it.
Then, in 1968 President Richard Nixon took office.
In the 3 year period before President Nixon took office Martin Luther King, Robert Kennedy, and Malcolm X were all assassinated, and riots occurred in Newark, Baltimore, Memphis, and Los Angeles. Combined with President Nixon promising to end the Vietnam War, the country was extremely fragile. In 1970 everything seemed to come to a head. 2 major events occurred in 1970 which have set the precedent for the last 50 years. First, the Kent State shooting in May 1970. The details are available elsewhere but this moment was the beginning of the end. The Kent State shooting, combined with President Nixon's response (or lack thereof) instilled the sentiment that the police were not your friendly neighborhood watchmen, but rather a force meant to keep citizens under control. A few months later, President Nixon passed the Comprehensive Drug Abuse Prevention and Control Act of 1970. The Act included the Control Substances Act which is the legal foundation for every single "fight" or "war" against drugs since. It is widely seen and credited with the largest expansion of the American prison system and the mass incarceration problem we're still dealing with today.
After Vietnam and Nixon, the country "self-corrected" and the economic boom of the 80s helped calm tensions. However, the legal impacts of the late 60s and 70s were too established. The Report from 1967 and Nixon's 1970 Act were used as the basis for the Anti-Drug Abuse Act of 1986 which is the infamous War on Drugs by President Reagan. The Act, among other things, enacted a variety of mandatory minimum sentencing rules. This continued into President HW Bush's 4 years where he passed the National Defense Authorization Act of 1990. That authorized the transfer of military hardware from the Dept. of Defense to federal and state agencies to help combat the War on Drugs. Boiled down, it essentially meant that local law enforcement had military grade weapons and equipment to police its citizens. This led to the Violent Crime Control and Law Enforcement Act of 1994, passed by President Clinton. That Act was the largest crime bill in history and provided for 100,000 new police offers, almost 10 billion in federal funding for prisons, and the infamous "three-strikes" rule.
The 1967 Report and the first Act on Drugs in 1970 all set the precedent for subsequent acts, as well as the dramatic increase in prison population. All of this essentially has boiled down to a simple point that isn't often talked about - the entire legal system (courts, police officers, and prisons) are incentivized to punish. Police officers are incentivized to arrest people, courts are incentivized to convict and sentence people, and the prison system is incentivized to keep people in. This has led to where we are today.
EDIT: I keep getting a similar comment so I want to address it here. I tried to make the focus of my answer the impact legislation has had on our perception of the police force since that is my background and expertise. The Kent State shooting was mentioned simply because it led to the passing of 2 of the most important crime bills in our history, The RICO Act (which led to no-knock warrants and expansion of civil forfeiture) and the Controlled Substances Act (the basis for every war on drugs). My intention was not to say the Kent State shooting was done by a police force. Simply put, the shooting led to the consolidation of power for policing and greatly expanded the capabilities of the local police force to control its citizens, hence the end of your friendly neighborhood watchman.
As for the militarization of police forces. That is a very related subject and no doubt has led to a shift in attitude and perception but I cannot speak to that subject. Others here have mentioned changes in training and other militarization aspects, and I will defer to them. I only meant to tangentially refer to it as it relates to laws enacted and the legislative impact.
The British police can really be said to have been founded in 1829, when the Metropolitan Police Act, at the instigation of Peel, set up the Police Office for London and defined the powers and duties of the Metropolitan Police. These were initially wide - the Act ordered each constable to arrest ;all loose, idle and disorderly Persons whom he shall find disturbing the public Peace, or whom he shall have just Cause to suspect of any evil Designs, and all Persons whom he shall find between sunset and the Hour of Eight in the Forenoon lying in any Highway, Yard, or other Place, or loitering therein, and not giving a satisfactory Account of themselves'. The proposition of setting up a police body met huge resistance from large sections of the public - an 1830 handbill was passed around calling them 'Raw Lobsters - Blue Devils' [1].
The date of 1829 is important, because Britain was at war with France, more or less continuously, from 1789 to 1815, so was only fourteen years out of almost a thirty-year war that had taken up most of the adult lives of most of the population. Much of this opposition was a reaction to what was and had been happening in France. France had a highly effective police force, which had stemmed out of the National Guard in Paris and had been used by Napoleon as a major arm of social control (not for nothing has a fairly recent study talked of 'the Napoleonic police state' ) and intelligence. Sibalis reckons that 'thousands' of men and women were arrested for opposing the French state, often only for critical remarks or 'on suspicion' of wrongdoing. Indeed, one of Napoleon's prefects wrote later that 'the imperial police ... was arbitrary, that was in its nature, and that's why in free countries people disapprove of a so-called 'ministry of general police'' - and was defending the police when he did so, on the grounds that they were better than the police in 'many established countries' (by which, one assumes, he largely meant Russia) [2].
With that in mind, Peel established the Police as a strictly non-military body, under a model that is now known as 'policing by consent'. The most obvious manifestations of this were threefold - the uniform was blue (a strong contrast to a soldier's red), the ranks were non-military (with the exception of 'sergeant' - so the badge worn by an army lieutenant denotes a police 'inspector', three pips mark an army 'captain' or a police 'chief inspector', a crown marks an army 'major' but a police 'superintendent', and so on), and the police were usually unarmed - a cutlass could be issued for riot duty, but normally only a truncheon was carried.
This ethos was codified into what are often called the 'Peelian Principles' - nine rules issued to each police officer in their booklet of 'General Instructions'. Though Peel talked of them in their speeches, recent work by Lentz and Chaires [3] suggests that the actual formulation was the work of the first two Commissioners, Charles Rowan and Richard Mayne. A few of those bear repeating as signs of what Peel and the early Met were trying not to become:
To recognise always that the power of the police to fulfil their functions and duties is dependent on public approval of their existence, actions andbehaviour, and on their ability to secure and maintain public respect.
To recognise always that to secure and maintain the respect and approval of the public means also the securing of the willing co-operation of the public in the task of securing observance of laws.
To recognise always that the extent to which the co-operation of the public can be secured diminishes proportionately the necessity of the use of physical force and compulsion for achieving police objectives.
To use physical force only when the exercise of persuasion, advice and warning is found to be insufficient to obtain public co-operation ... and to use only the minimum degree of physical force which is necessary on any particular occasion for achieving a police objective.
To maintain at all times a relationship with the public that gives reality to the historic tradition that the police are the public and that the public are the police, the police being only members of the public who are paid to give full-time attention to duties which are incumbent on every citizen in the interests of community welfare and existence.
It is not difficult to see the image of the imperial police force, in the negative, in those statements. This is the tradition on which all British policing is founded, and it has kept the same to this day - though the Home Office felt obliged to gloss, in 2012, that 'policing by consent' does not mean that you have to consent in order to be policed.
Obviously, it goes without saying that 1829 is a fair while after the foundation of the United States, so there was no real reason why this attitude should transfer across the Atlantic. Nor do the unique historical (that is, anti-French) ideological conditions of the founding of the Met apply to the United States. The US police came out of for-profit organisations and only existed on an informal level while all this was going on. City police forces in the US are really a feature of the 1880s and later, and a reaction to fears of immigration from Europe and of labour unrest - strongly linked in to local party machines, and therefore the people who controlled them and their networks (based in the capitalist world) - and therefore pretty explicitly a means of class control. To a large extent, the history of policing in the United States has been a story of government trying to rein them back from that place.
So the two organisations come out of completely different situations and conceptions - it should hardly be a surprise that they developed different cultures.
Sources
[1] You can find a low-resolution picture here ( http://www.open.ac.uk/Arts/history-from-police-archives/Met6Kt/MetHistory/mhOppPol.html )
[2] Most of this is from Michael Sibalis' chapter in Philip Dwyers' edited volume' Napoleon and Europe' (2001) bears this title.
[3] 'The Invention of Peel's Principles: A Study of Policing "Textbook" History". Journal of Criminal Justice. 35 (1) (2007): 69–79.