This is a complex question and not really in the realm of US legal history that I have spent the most time in, so I will try to outline the specific limitations of what I am saying as I go along. The short answer is that as part of the systematization of mass incarceration in the 19th century, the United States did initiate a period generally recognized as oriented around rehabilitation when large-scale efforts were undertaken; there are several important caveats to this that I will try to cover shortly. The end of that period is often held to be the 1970s and 1980s, as part of the conservative resurgence that would begin with Nixon (at the national level) and culminate in the Reagan "revolution", at which point "tough on crime" could be regarded as the new middle ground in American politics. A useful marker is Robert Martinson's 1974 paper ‘What Works? – Questions and Answers about Prison Reform’, aptly summarized as the birth of the scholarly "nothing works" doctrine. While it would be criticized in short order by other scholars and eventually largely withdrawn and corrected by Martinson himself by 1979, the belief that rehabilitation was a failure and could be backed by science became fairly dominant.

Over this entire discussion is the fundamental problem of states vs other states and vs federal law; in some ways, this renders discussion of an "American" prison system difficult even with relatively recent history, but in other ways I think your question does get at the heart of some useful generalizations we can make. Just understand that addressing it in this manner is a useful conceit, one that provides us with a model that is necessarily inaccurate both in terms of the ebb and flow and the overall practices in a given state or region.

Earlier, I mentioned the caveats to the notion of a progressive period. First, the markers used for this are the large prisons at Auburn, Sing Sing, etc, built in the 1820s and afterwards. Paradoxically (given the terms of your question) the intent was to isolate inmates from society in order to rehabilitate them. This, in addition to the many other potentially problematic "rehabilitation methods" of the period, mean that for the modern observer there is a clear tension between intent of the designers and practice as experienced by the inmates. So to look more specifically at how this works: The Sing Sing Correctional facility was regarded as both modern (in keeping with the times, it would turn a profit for the state from prisoner labor) and rehabilitation-oriented (through the Auburn method, regarded as cutting edge means of making bad people behave themselves). Neither of those would likely fit your modern definition of rehabilitation-oriented, particularly as the Auburn method was essentially universal solitary confinement (at night) and flogging-enforced (as well as all kinds of more subtly brutal practices, once flogging went out of style) silence during the workday. And yet in WWI-post WWI era, Sing Sing would in fact transition to something more recognizably rehabilitation-oriented, as a couple of wardens radically changed the prison to match early 20th century progressive ideals, which can still be criticized for all kinds of reasons but were an undeniable improvement in quality of life for inmates (among other things, sports leagues, education, etc). So while that's just one prison, I think it's fair to say that while rehabilitation outcomes were often questionable from the birth of large prisons to the 1970s, the intent of reformers was at least nominally that prisoners be rehabilitated.

Second, there's the problem of race, Jim Crow, lynching, and a variety of related factors. Lynching in America is a useful source here, as it surveys a number of scholarly works in order to provide some interesting and compelling claims about American justice. It's a very dense (in a good way) piece, but my summary would be that in parallel with any and all progressive elements during the birth and evolution of mass incarceration, you had a vast extralegal/vigilante brand of "pure punishment/vengeance" doctrine in action along primarily racially coded lines in lynching, and it was both popular and tacitly approved, at least at the local level, by authorities. Most interestingly, he alleges that the systematization and standardization of the death penalty in the first half of the 20th century was an effort to co-opt murderous, racist vigilantism as much as anything else, which suggests that in at least a few important ways that for blacks and other minorities the progressive era of prison never really happened. While they are legal scholars and not historians per se, The New Jim Crow and The Collapse of American Criminal Justice both provide compelling analyses from different legal perspectives for the role of rehabilitation relative to different groups and why that occurred. Finally, from a historical perspective, Texas Tough provides an interesting argument that the Jim Crow-rooted Southern approach to punitive justice both endured throughout the so-called progressive period and would eventually become the national model in the 70s and 80s. That "universalization" of punitive prison practices is, I think, another useful argument for a significant change even if it inherently refutes the notion of a progressive, rehabilition-oriented American consensus.

Thirdly, there is the question of design/intent versus systemic outcomes that may be beyond any particular goals or strategies. That's a good place to talk about the stresses created by the American war on drugs and other policies that derived from "nothing works"-related beliefs in the incorrigibility of criminals and the threat that "soft" punishments would make them the pace-setters of society. In short, from Nixon's declaration of the war onwards, there's a steady upward trend in the raw number of people targeted for incarceration, often in ideological climates where a concomitant expansion in legal resources pre-incarceration and facilities/strategies during and after incarceration. In short, I would suggest that the raw increase in inmates, coupled with extensive racial and class coding of felon/ex-convict status concentrations in society, provide a systemic feedback loop that subsequent forays into privatization have only enhanced in favor of punishment. There's sort of a chicken and egg problem going, then, where you have to isolate different points in different places and decide which is driving harsher regimes (ideology or numbers) at different times.

There's a lot that I'm glossing over here, but I hope the general impression of a shift in the 70s and 80s is a useful one.

Alexander, Michelle, and Cornel West. The New Jim Crow. The New Press, 2012.

Miller, Randall M. “Lynching in America: Some Context and a Few Comments.” Pennsylvania History 72, no. 3 (October 1, 2005): 275–91.

Perkinson, Robert. Texas Tough: The Rise of America’s Prison Empire. First Edition edition. New York: Picador, 2010.

Stuntz, William J. The Collapse of American Criminal Justice. Harvard University Press, 2011.