There were those who claimed Confederate loyalties while pursuing individual reward, like Archie Clement or Jesse James. It's a fool's game to draw a sharp line between absolute criminality and absolute political motivation, but there are plenty of examples of "outlaws" who wrapped themselves in the Stars and Bars.

Once Reconstruction began, there was a sort of shadow government in the South, a network of conservative ex-Confederates who coordinated the long terrorist campaign of the Ku Klux Klan. The KKK and associated groups weren't an official army, and most of its actions were driven by local events rather than any coordinated strategy, but they conducted thousands of terrorist acts against federal troops and the mostly black citizens who supported the federal government. As a matter of fact, on several occasions elected governments were toppled - most notably when a local militia, the White League of Louisiana, seized control of New Orleans in 1872. The coup collapsed three days later when federal troops arrived in the city.

What I think you're looking for, however, are actual Confederate units that refused to disband. Several thousand Confederates fled to Mexico at the end of the war, most notably General Joseph Shelby, who led his unit of 1,000 cavalry in order over the Rio Grande. Most other Confederate refugees fled as individuals or in small groups.

Stand Watie, Cherokee General in Indian Territory, who pretty had guerilla fighters, was doing pretty good on his own. Just killing there, so he didn't surrender with the rest of the army and Lee. He did however surrender a few months(I think, correct me if I'm wrong) after the war.