No, it was seen as a Union holiday and after the war was an event mainly celebrated by freedmen. The worst lynchings and massacres in Reconstruction occurred when white people attacked 4th of July celebrations.

There's not a specific date when Southerners began to reconcile themselves with the Union but most historians set the beginning of reconciliation to when Rutherford Hayes was elected President. Generally the Spanish-American war around 1898 is a safe bet for when most people in the South would think of themselves as part of the Union as Nationalism began to rise in politics.

Source: Race and Reunion by David Blight

I have a follow up question: did the Confederacy have their own equivalent to the 4th of July?