The only specific case that I know of is Virginia. When Virginia voted to secede, the western third or so of the state was not enthusiastic about it, and the government in Richmond wasn't strong enough to compel anybody there to follow them. This region is today the state of West Virginia. There was a Unionist government in Wheeling that later transitioned into being the government of West Virginia. When West Virginia entered the Union as a state, a Virginia government-in-exile of sorts convened in Alexandria, Virginia just across the river from Washington, DC. This government claimed authority over the entire state. As a practical matter, eastern Virginia was a war zone for virtually the entire war, so this civilian government necessarily took a back seat to whatever the army was doing.

The state of Missouri didn't have quite the same situation, but was interesting in its own way. Missouri, like other slave states, called to order a convention to vote on seceding from the Union. The Missouri convention voted resoundingly against secession, but the pro-Confederate governor refused to call out state troops to fight secession. This prompted the Missouri Convention to declare the offices of state vacant, and the convention acted as a sort of provisional government for the state through 1863, with a separate Confederate government-in-exile operating out of Texas.

Readings

For the Civil War era generally, see McPherson's Battle Cry of Freedom

For Virginia during this period, see Richard Lowe's Republicans and Reconstruction in Virginia, 1856 - 1870

If you're wondering about Unionist governments that operated at the same time as Confederate ones, I'm not aware of any of those. My area of focus is East Tennessee, and also some of Western North Carolina, so it's possible that these existed in other places and I just haven't heard about them.

However, there was an attempt on the part of East Tennessee to leave Tennessee and return to the Union. It was a secession from the secession, if you will. It was called the East Tennessee Convention, and I mean Convention in the noun sense rather than the verb because this group actually gathered three times throughout the war: twice in 1861 and once in 1864. The 1861 Conventions debated for awhile and, in order to not stoke passions further and cause bloodshed, rather than seceding outright they ended up petitioning the state legislature for the right to secede from Tennessee. But Confederate troops occupied the area before anything came of it, and after that the government depended on which side had wrested military control. Those who were on the opposite side of those in charge didn't seem to organize so much as they worked to protect one another from retribution, which was a frequent event. The Convention didn't meet from 1861-1864, so it didn't exist as a functioning shadow government per se, though it did technically still exist during those years.

Check out "The Civil War along Tennessee's Cumberland Plateau" by Aaron Astor.