There are pages of book recommendations in the FAQ, including the "Civil War / Reconstruction Era". (In the app, it's maybe 2/3 of the way down the page.) The section before is "Antebellum Slavery". Each book usually gets several sentences of review.

But this is not to discourage more suggestions -- I hope we get more.

I'll echo the suggestion of Battle Cry of Freedom: The Civil War Era by James McPherson. I like it because firing on Fort Sumter happens on p. 273 of 867 -- the pages before it covers years of background with the many factors that led up to it. Also, it is nowhere near just a military narration. He covers a lot of politics (government and opposition), and also finance, economics, trade, non-Civil War items, and a multitude of other non-military topics.

Edit: Those other notes don't derail or bog down the narrative. Those other parts often are a few pages giving a brief overview of what happened and its effects. For example, there are a few pages on how how the United States and the Confederacy respectively chose to finance their wars in different ways, and the effects. Or unrelated Republican laws.

Shelby Foote's three volume history sounds perfect for you.

I'll have to go with Hardtack by Kelsey Brickl. It's much grittier than most romances and focuses a lot on the pre-war experiences of the characters, too. I also appreciate how much detail Brickl goes into with a particular character's experience with amputation. I actually know an amputee who said the portrayal was really, really accurate. It's a quick read and is historical fiction, but it does cover some military history and it's not a "girly romance." I def recommend it.