I've read the book "The Bill Of Rights: Creation to Reconstruction" by Akhil Reed Amar that covers the drafting of the 14th Amendment, what John Bingham was trying to do and the reaction of the courts in cases like Slaughter-House and US v. Cruikshank. I intend to read "The Day Freedom Died" by Charles Lane on the events at Colfax LA and the Cruikshank case that came out of it.
I guess the answer is really "we were still a bunch of racist fucks" but...I think there's lessons here to be learned about an entire civil rights movement that failed hard...
I honestly believe that it was much more war fatigue then anything else. Lincoln was followed by Andrew Johnson, a Democrat. Johnson, in my opinion, did more to harm civil rights in the United States then any individual in our history. As president he ordered the southern states to reform their governments, returned confederates to power(who Johnson had pardoned), and allowed the reconstituted governments to pass black codes, which basically returned the vast majority of negro's into a position similar to their pre-war state. He also opposed the 14th amendment, and worked hard to remove Lincon's loyalists, which ultimately resulted in impeachment.
Johnson was succeeded by Grant. Grant supported radical reconstruction, and began to rollback Johnson's changes. This led to a high level of violence, mob lynchings and assassinations. In a very real sense, it was similar to what we see in parts of the middle east with modern sectarian violence. Black voters came out in force, and African-American's who elected to all levels of government. Northern opportunists came down to the south, and rightly or wrongly, where identified as corrupt "carpet baggers".
Grant's presidency is marred by massive corruption, and the first major depression that occurred in 1873 caused the administration to lose support in America.
The KKK, which was to the Democratic party as the IRA was to Sinn Fein, continues to escalate violence, including killing Republican and African-American lawmakers in the south while the northern states increasingly wanted to disengage from the "southern problem", no matter what the outcome for African Americans would be. Democrats returned to control the house in 1874, ending legislative initiatives to protect Negros, and continue reconstruction. Two horrible US Supreme Court cases further restricted the federal government's ability to protect citizens.
Finally in one of the most corrupt deals ever - the Republican and Democratic candidates in 1876 deadlocked in the electoral college. The Democrats gave the Republicans the presidency in exchange for ending reconstruction, locking in Jim Crow for another hundred years.
tl;dr The KKK, which was the military wing of the southern democratic forces, unleashed a decade of violence, while Republican forces collapsed to corruption and depression. The bad guys won.