For an absolute tour de force answer read McPherson’s Battle Cry of Freedom pages 276-277 & 284-298. It answers this question perfectly.

While none of these states seceded from the Union they all had residents who fought on behalf of the Confederacy to varying degrees. In an interesting fact, Missouri and Kentucky were admitted into the Confederacy as the twelfth and thirteenth states. But historians don’t look at these states as having officially seceded because the Union maintained political control of their state capitals and the majority of territory.

Delaware: There was almost no possibility Delaware was going to secede from the Union. “Less than 2 percent of its people were slaves, and more than 90 percent of its black population was free.” In January of 1861 the state legislature expressed a vehement disapproval of secession; the question, for all intents and purposes, was put to rest before the first shots of Ft. Sumter.

Kentucky: Expressed neutrality in the beginning of the war. The state was bordered by three slave states and three free states. It’s noted, “Kentuckians took pride in their traditional role as mediator between North and South.”

Lincoln accepted neutrality for a short time and went so far as to allow supplies to flow between Kentucky and Tennessee. His prudence turned out to be the right decision. When Gen. Polk invaded Kentucky on behalf of the Confederacy in September 1861 he “converted the legislature from lukewarm to warlike Unionism.” This left the Pro-Confederate Governor Magoffin and Senator Breckinridge no option but to resign and take up their lot with the Confederacy.

It’s very possible the state could have gone to the Confederacy but Lincoln played the right cards by letting the rebels attack first--they were viewed as the defenders, not the aggressors. The state would stay divided throughout the war and ranks of both Union and Confederate Army’s would be filled with the sons of Kentucky who would meet each on battlefields for years to come.

In November Magoffin called a convention and formed a provisional government to pass secession for the state. But again without exercising control over the capital and the state in general, this was seen as a political move that carried no real weight.

Missouri: The last two states, Maryland and Missouri, came the closest to secession. Governor Claiborne Fox Jackson represented the Pro-Confederate faction in Missouri and on the Pro-Union side stood Congressman Francis P. Blair Jr. with Nathaniel Lyon as his military counterpart.

In Jackson’s first address as governor, he advocated that the state should, “bind together in one brotherhood the States of the South.” After Ft. Sumter, “Jackson took control of the St. Louis police and mobilized units of the pro-southern state militia, which seized [a] small U.S. arsenal.” He requested Confederate artillery, received it from Jefferson Davis under the guise that it was "marble", and began drilling the militia at the aptly named Camp Jackson.

But Blair, with Nathaniel Lyon taking the lead, countered every move that Jackson made. They formed several pro-Union regiments in support of the federal government. And deciding to press the action, Lyon masqueraded as a woman to reconnoiter Camp Jackson. The next day Lyon surrounded Camp Jackson with his regiments; all the members of the militia surrendered. But here’s where his plan almost backfired. Lyon met stiff resistance from the civilian populace while marching the prisoners through St. Louis. Shots were fired and “twenty-eight civilians and two soldiers lay dead or dying, with uncounted scores wounded.”

This incident incited rage at the state capital in Jefferson City and pushed many legislators into Jackson’s favor as well as prompting many “conditional unionists” into the ranks of the Confederacy. Peace talks were brokered between the two warring parties to squelch a potential state war but ended with Lyon declaring to Jackson that it was war!

Lyon was a man of his word and occupied the capital in Jefferson City, a few days later, with no fight from the militia who were posted there. Lyon used force instead of diplomacy to drive out the pro-Confederate state government. But he refused to rest on his laurels and pressed the action, driving the militia to Missouri’s southwest border. This did have consequences as it stirred up guerrilla warfare throughout the state but had the strategic impact of establishing Union political control.

So while Lincoln was able to gain control of Kentucky through patience & politics, Lyon did the exact opposite and pushed out the pro-Confederate governor of Missouri through might & military force--Lyon became the Union’s first war hero.

Governor Jackson and a large majority of the legislature had “decamped” so in July the only remaining government--the pro-Union state convention who had already rejected secession--assembled and elected a new governor and filled other vacated government positions. “The convention ruled Missouri until January 1865, when a government elected under a new free-state constitution took over.”

Jackson did get Missouri admitted to the Confederacy in November, the same month as Kentucky, of 1861 but they were driven from the state shortly after seceding and spent the rest of the war outside the borders of Missouri.

OP, since you’re from Maryland I’m not sure if you wanted more background on it. If so please post in the comments. I’m off to bed!

Maryland Southerner here. Almost nowhere in the US was more divided than this state. At the very beginning of the war, pro-Confederate mobs violently attacked Yankee troops marching through Baltimore. Lincoln declared martial law in Maryland a little bit later, if memory serves me correctly, and the staunchly pro-Confederate mayor of Balitmore was arrested.

The Eastern Shore is a whole different story. We have always had a strongly "Southern" culture here, despite our being isolated from the rest of the South by the Chesapeake and our long border with the Mason Dixon line. Here on the Eastern Shore, Union troops were placed to dispel secessionist mobs. I honestly think that the Eastern Shore would have seceded from Maryland and joined the Confederacy if our population had been a bit larger at the time. An entire regiment of Maryland Confederate troops was raised here, and sent to Virginia to serve in the Stonewall Brigade. Here in Easton, we have a statue called "To the Talbot Boys", memorializing the Confederate Soldiers who fought from Talbot County, including the aide-de-camp to Gen. Washington, Tench Tilghman's, grandson. Similar statues are found throughout the Eastern Shore, in towns such as Elkton and Chestertown. Anecdotal reports note that Easton had a few small fistfight-turned shootouts between Yankees and Confederate sympathizers.

It is worth noting that parts of Western Maryland, especially around Frederick, were staunchly pro-Union. When Robert E. Lee marched up from Virginia, his troops expected to find pro-Confederate support throughout the state. Instead, in the western section of the state, they were met with ill will and malice.