Adam Good heart's book "1861" confirms that some Confederates celebrated the 4th of July. For example, on July 4th, 1861, federals at Fortress Monroe, VA thought they were under attack, but it turned out to be a cannon salute by confederates for Independence Day.
This practice makes sense within the context with which the southerners approached their secession. Rather than southerners betraying the union, it was the North who had gone against the principles of state's rights and the ability to keep property (I.e. slaves).
The New Orleans Daily Picayune wrote “The people of the Confederate States of the South alone remain loyal to the principles of the Revolution...To them now belongs of right the custody of all the hopes of human progress, of which the Fourth of July is the symbol in history, and it is by their swords that it is to be saved for mankind.”
A stark exception to Confederates celebrating the 4th of July was at Vicksburg. According to Winston Groom, in his book "Vicksburg 1863," after the surrender of Vicksburg on July 4th, 1863, the city did not celebrate the 4th of July again until 1945.
I'm not sure if this deviates too far from OP's question, but did the Confederate States celebrate their own Independence Day? If so, what day was it? I understand the different states declared their independence / succession at different times.
Yes, many people in the Confederate states celebrated the 4th in the beginning of the war. However, from what I've been able to find, the celebrations died down to the point that they could be basically overlooked later in the war. One of the best examples I've seen of this is if you look at newspapers from July 1861, announcements of celebrations and articles remembering the day can be found throughout the paper, where you'd have scour the articles to find so much as an off-handed mention of it in later years.
For example:
The Memphis Appeal - July 4, 1861 - containing a good bit of mention of the day and celebrations: http://chroniclingamerica.loc.gov/lccn/sn83045160/1861-07-04/ed-1/seq-2/
The Memphis Appeal - July 4, 1863 - no mention of it that I can find in this or the following issue from the 6th: http://chroniclingamerica.loc.gov/lccn/sn83045160/1863-07-04/ed-1/seq-1/
American Citizen (Canton, MS) - July 4, 1861 - mentions 4th of July Celebrations: http://chroniclingamerica.loc.gov/lccn/sn83016736/1861-07-06/ed-1/seq-2/
I can't find any mentions in the American Citizen of celebrations around the 4th in 1862 or 1863. Likewise, I've yet to see mention of July 4th celebrations/remembrances in any other pro-Confederate papers from about 1863 on. This could be due to a focus on the war and developments around it, but other less-important stories are carried in them during that time frame, so it seems odd that they never mention the the 4th of July unless it had ceased to be widely observed.
More evidence of a possible anti-4th of July sentiment among Confederates is the fact that you can find multiple mentions of the date during the war in pro-Union southern newspapers at the same time, like the Nashville Daily Union from July, 1863.