Related question, were the so called "hanging judges" of the American Old West a real thing, and were they within their legal rights to deal out executions as the primary form of punishment?

According to Foucault's Discipline and Punish, brutal public executions and bodily mutilation used to be the norm. Prison emerged as an alternative idea to reform the prisoners, and gradually took over since it fit well with the modern state's tendency to exercise power bureaucratically, and categorize its citizens (into categories such as law-abiding, prisoner, felon) in order to control them.

[sorry if Foucault is not a AskHistorians grade source]

In the United States, it was the 1820s. Pennsylvania and New York were the leaders, developing the "separate system" and "silent system," respectively. Under the first, prisoners were kept alone in their cells in total isolation for their entire sentence, while under the second, prisoners were let out of their cells during the day for meals and work and such but weren't allowed to talk. Eastern State Penitentiary in Philadelphia is considered to be the first real prison in the U.S., and is a museum now. Check it out if you're ever in Philly. The separate system was developed at Auburn a Prison in New York, which is still in use as a prison.

And yes, corporal and capital punishment were both more prevalent in colonial and immediately post-revolutionary America, carried over from English law. Imprisonment was a development thought of as a more humane alternative and as one that gave a chance for rehabilitating people rather than just punishing their bodies. Large standing jail systems were not a thing before the 1820s.

The English transported a lot of criminals to America before the Revolution, and started using Australia after. However, they were influenced by the developments in the U.S. too. Pentonville Prison was built on the separate system, and prisoners there had to wear masks when they left their cells.

Incarceration as we know it is a product of enlightenment thought. Prior to the enlightenment people accused of crimes that did not merrit execution were either fined and or physically punished or even exiled. This was done publicly infront of the community. A quick note on exile and the term outlaw. An outlaw is someone who is outside of the law and not afforded its protections such as a trial, it means that you can be (and should) be killed on sight, a translation of the original phrase means "to have a wolf's head" the implication being that if seen such a person should be killed on sight. The enlightenment brought about the belief that man is fundamentally good and is capable of reform, and what better way to reform ones self than through solace and contemplation. This is where the one room cell comes from. I can't say at what point in the 18th century incarceration began to replace physical punishment, but I know for sure that the Bastille was being used as a prison in Paris in the decade before the start of the revolution, so by the mid to late 18th century we had people being jailed.