Well, I don't think it's at all fair to call Kentucky a "backwater," but I get your meaning, and I'll try to explain Kentucky's development vis a vis the rest of the nation.

I would first argue that Kentucky wasn't in all that great shape in 1860. There had not been much development in the Commonwealth during the antebellum decades. There was no interstate railway. Most people were farmers living at the subsistence level, and the mechanization of agriculture was not keeping pace with other states. Urbanization was sluggish. There had been little exploitation of timber or mineral resources.

Following the war, Kentucky politics came under the control of former Confederates who were interested in erasing all Radical Republican accomplishments and resisting economic and social change. They wanted to keep Kentucky agrarian. They and their ideological descendants held onto power well into the 20th century. A new state constitution in 1891 was almost impossible to amend and made it difficult for local governments to raise tax money and change society. Its purpose was to maintain the status quo. Calls for a new constitutional convention throughout the 20th century were shot down.

This all set the stage for a relative decline during the Gilded Age. The Civil War wrecked what railroads there were. It also destroyed the leading market for Kentucky's products--the Lower South. European immigrants settled in California, not Kentucky. Efforts to attract immigrants did not work, and population was stagnant.

There was no public school system in the state. Public schooling had only gotten underway in the 1850s, and the Civil War disrupted the tax base and teacher training. This perpetuated the high level of illiteracy in the state, and made it impossible to expand the economic base beyond agriculture. Officeholders were not concerned about this state of affairs.

By 1900, then, while there had been some railroad construction, some manufacturing growth, and some important efforts to exploit natural resources, especially coal, the state remained incredibly agrarian compared to the rest of the nation. Much of the land remained undeveloped. In 1880, 18% of American workers were farmers, but 68% of Kentuckians. In 1940 only 12% of Kentuckians worked in factories. The political leaders of the state, afraid of social upheaval, did not want to enter the modern world.

The Progressive Era brought some changes. There were major accomplishments in education reform. There were improvements to infrastructure. However, it was only after World War I that we see a "social and economic revolution" in the state thanks to the introduction of the automobile. We finally see a real rise in industrialization, a decline in agriculture, and attacks on illiteracy. But the state would need to play catch-up with the rest of the country, and do so with a constitution that was intended to keep the Commonwealth in the 19th century. It has made it extremely difficult for local governments to raise revenue and enact economic improvements.

Tl;dr: postwar state leadership wanted it this way.

And in terms of culture, well, Hunter S. Thompson was from Louisville....

Source: Thomas D. Clark's introduction to The Kentucky Encyclopedia and some other stuff.