A tiny fraction: a few hundred active duty officers from the pre-war US Army and Navy. Very few enlisted men went over to the Confederacy. Many were foreign-born, most had little loyalty to the south, and none were at liberty to resign their commissions like the officers.

The second part of your question is the more interesting one, in my opinion. The Confederacy did not inherit an army, or even much in the way of the makings of one. The Confederate government raised an army in the same way the Union did: by demanding regiments of volunteers from the states for service. These men were the militia, but most were not trained and they can't really said to have been soldiers prior to entering the Confederate Army. Those few hundred professional officers, assisted by retired West Pointers, military college graduates, and volunteers of varying degrees of competence, then had to turn these volunteers into soldiers, first in drill camps, then in the field.

I should add that in April 1862 the Confederate Congress passed the Conscription Act, which introduced mandatory military service for able-bodied white men of military age as well as extending all enlistments for the duration of the war. But the initial army that took the field was composed entirely of volunteers.