Northern loyalists, you might say?
This older answer should be of interest.
You may be interested in this earlier question Where their [sic] any pro-union groups of guerrillas operating in the south during the Civil War? which got answers from /u/eternalkerri and others.