It seems hard to believe that the United States defeated the Confederacy without there being some Confederate militants/rebels resisting the outcome of the war. Were there any anti-Union prostests or riots in the following years? How long was this a problem?
In a word: abso-frickin-lutely. Even if we take out the ongoing problem of African American lynching, voter intimidation, and general disenfranchisement that was rooted in systematic, organized violence ALL throughout the late-19th and 20th century, we can point to several specific instances of civil unrest.
The most famous instances were probably the New Orleans riot of 1866, and the Colfax Massacre of 1872, which was the result of actions taken by entrenched Confederate sympathizers under the guise of the Democratic establishment. The N.O. riot of 1866 centered around the reconvening of the state constitutional convention, which passed resolutions allowing African Americans to vote. This didn't sit well with the local Democratic (ex-Confederate) legislators, who pushed to block these reforms on a series of technicalities. Long story short, African Americans marched in protest of this blockage, and all hell broke loose. On their way to the state convention to debate/vote on these blocks, the African Americans were savagely beaten by Democrats opposed to the expanded voting rights (roughly 3 dozen killed, hundreds injured).
This was nothing compared to the Colfax Massacre, however, which took place a few years later in Louisiana. The fight against African American voting and civil rights had never really died down, and by some estimates, anywhere between 1000-1200 African Americans in Louisiana were murdered in this period between 1866 - 1872. There was a straight-up, organized, coordinated effort to attack the Grant Parish Courthouse in Colfax, where Republicans had organized to protect local African Americans and guarantee their ability to register to vote without violent interference. The courthouse was also symbolic, and represented Republican control over the local government (control that was being used to protect the civil rights of the African Americans there). The Democrats (ex-Confederates) spread rumors of insurrection, rape, and murder, and got several hundred volunteers and a cannon organized to attack the courthouse and the African Americans there. Estimates range, but most agree that roughly one hundred African Americans were killed.
What's worse, however, is that the Supreme Court ruled that the government could only enforce civil rights legislation when violations of the 14th amendment were perpetrated by the state. If it was individuals violating civil rights, the states, not the government, had to prosecute (and surprise, surprise: Louisiana's state courts didn't prosecute the individuals). This was a big reason why Democrats were able to control state governments for so long: no one dared oppose them, as the people doing the intimidating/killing weren't getting prosecuted by the friendly state governments.
There were similar instances all throughout the south, in Mississippi, Texas, etc. In all, it was really, really, really bad for about 100 years, until Johnson passed the Civil Rights Act of 1964. Obviously, as a country, the U.S. is still sorting through a lot of this violence and discrimination (see Charlottesville, VA this weekend), but for the purposes of OP's question, hopefully this will suffice.
[Sources - Eric Foner, 'Reconstruction: America's Unfinished Revolution'; James Hogue, 'The Battle of Colfax'; Ronald C. White, 'American Ulysses'