Michel Foucault offers a reasonably convincing argument for why this is the case in Discipline and Punish which links the rise of imprisonment in Europe to the rise of what he refers to as a disciplinary culture. I haven't looked over it recently enough to give the argument it's full due, but basically, prior to the 1700's, imprisonment was quite uncommon in western europe; you might imprison someone before you punished them (possibly interspersed with some torture, especially on the continent), but to do so as the punishment was generally seen as inefficient.

Then, he argues, efforts towards penal reform took hold during the 1700's, and especially after the french revolution, due to what many people saw as an inefficient distribution of the power to punish: some crimes were punished too severely, some not severly enough, the court systems were often quite decentralized, public punishments not infrequently led to riots, and so on. By creating more efficient, enlightenment-style rationalist penal systems, reformers believed that the reform of criminals and public morality could more easily be accomplished.

However, these penal reformers tended to favor poetic justice-like punishments, in which somebody who, for example, committed a crime associated with pride would be publicly humiliated, people who committed crimes associated with being greedy would be fined, and so on. However, for a muddle of reasons having to do with the desire to centralize, view, and control criminals, prisons ended up becoming the standard throughout western Europe. Although they didn't eliminate criminality (or even really reduce it) as many had hoped, they did allow governments to manage it in new ways, sequestering them away from the ordinary population and gaining systematic knowledge about them, giving them criminal histories and keeping them out of, for example, the scaffold in the town square. On account of this, they were convenient solutions to a problem (although not exactly the problem that people thought they had) and became the standard throughout western Europe.

Of course, it was a long process towards imprisonment as common worldwide practice. Many countries picked up english or french-style legal systems due to colonization; others switched in an effort to reform cultural practices seen as barbaric (as arguably happened in China in the early 20th century). And of course, countries that practice strict interpretations of Sharia law still use bodily punishment for many more crimes than, say, the US or EU.

Perhaps somebody with the right books on them can provide better dates for the processes of penal reform in Europe.