While more can always be written, you may want to check out this post by /u/lord_mayor_of_reddit, which also references this earlier post by /u/chaingunfighter.

TL;DR:

Martin Van Buren was probably the most staunch Unionist of the bunch. He had fought to prevent secession before the war, and upon the attack on Fort Sumter by secessionists, he publicly called it treasonable. While he remained a Democrat, he supported the Republicans' military response. He could be accurately described as a "War Democrat".

John Tyler's actions were staunchly pro-Confederate and less "reluctant" than the brief comments by /u/chaingunfighter's about them might suggest. He at first helped organized the Washington Peace Conference of 1861, but when that conference failed, he immediately issued a statement in support of the Confederate position, and blamed the Union for the situation. Upon leaving the Peace Conference in Washington, D.C., he left for Richmond where he sat as a delegate at the Virginia Secession Convention, voting in favor of secession. Once Virginia seceded, he joined the Confederacy, becoming a member of the Confederate Congress.

Millard Fillmore and Franklin Pierce were privately of a similar mind, but there was still space between them, and publicly, there was a gulf between them. Pierce became a vocal Copperhead that denounced the actions of Lincoln and the Republican Party, and gave major public addresses to that effect. After the Emancipation Proclamation, Pierce even wrote that a war prosecuted to end slavery "is itself treason." Fillmore, on the other hand, was publicly a Unionist. In private, he criticized some of the Republicans' and Lincoln's actions, but was still steadfastly in favor of the U.S. and against the Confederate cause even in his private communications. Fillmore and Pierce exchanged some letters criticizing the war effort.

Pierce did try to organize a sort of "presidential peace conference" of ex-presidents after Fort Sumter, and wrote to Van Buren, suggesting Van Buren organize it, being the most senior among them. Van Buren wrote back basically saying, if you believe in it so much, why don't you organize it and publicly announce it. Pierce never followed through. Privately, Pierce offered some aid and comfort to Jefferson Davis, his Secretary of War during his Presidency.

James Buchanan made his views known in his final national addresses, most thoroughly in his Fourth Annual Message to Congress. He considered secession illegal, opposed the Confederacy, but believed it was a power left to Congress to declare any military action. He had been the one to appoint John Tyler a special envoy to Congress, which ultimately resulted in the failed Washington Peace Conference of 1861. He had supported peace negotiations throughout the Secession Crisis, lobbying Congressmen to support the various negotiations. To the end of his presidency, he hoped there would be a peaceful resolution.

Once Buchanan left office, he mostly stayed quiet on the matters in public, though privately, he supported the Republican effort to engage in the war. Like Van Buren, he could be characterized as a "War Democrat" if slower to make his views known. He publicly denounced the secessionists' actions at Fort Sumter as treasonable and a deliberate act to start a war. After the First Battle of Bull Run, he privately wrote that he believed "a vigorous prosecution of the war" was justified, finally going public that fall with an endorsement of Lincoln's war policy. Thereafter, he remained quiet in public about the war and the war effort. He instead spent the later war years writing his memoirs, published in 1866, which defended his own actions during his presidency, particularly his actions during the Secession Crisis.

There is a thorough retelling of the ex-presidents' actions during the war in the book The Presidents' War: Six American Presidents and the Civil War That Divided Them by Chris DeRose.

This answer i wrote about President Tyler might be of interest.