How would they decide the rank of someone at enlistment time in the Union in the Civil War?
There were two types of regiments in the Union Army during the Civil War: regulars and volunteers. The regular army was a standing force that existed prior to the war and was filled with men who had enlisted for a period of five years. The size of the army and the number of regiments was federally determined and was, at the start of the war, extremely small, both in comparison to most European armies and in the role it was expected to fill in and along the borders of the United States.
The vast majority of enlistees who enlisted during the war were volunteers, and legally distinct from the regulars. Depending on when they enlisted they might sign on for 90 days, or for increasing durations as the war went on. The regular units and volunteer units remained separate, and volunteer units were led by volunteer officers and volunteer NCOs.
Attaining rank was at once a simple process and a complex one; officers tended to come from the upper classes, tended to be wealthy or educated, and had likely had some officer-adjacent position either in a the local militia - which by the 1860s was typically organized as private regiments - or had social clout by virtue of their position in the community, or had some leadership role in fraternal organizations, like the masons, among others. Membership in a fraternal organization was extremely common for men of a certain class in the mid 19th century. The memoirs of Frederick Lyman Hitchcock, the adjutant of the 132nd Pennsylvania Volunteer regiment (their enlistment was for nine months), mention a time when Hitchcock and other officers were hosted at the home of a "secesh" Freemason, who shared his meagre food with the officers "on the strength of Freemasonry." This is all to say that officer positions were generally secured through class standing and social networking; Hitchcock was educated, a Mason, and known to other officers at the time the regiment was formed, and found his skills suitable for the position of adjutant, and so he was made so.
Enlisted men often lacked those kind of connections and social clout, but had their own internal social hierarchies that could determine their position, or regiment, or otherwise. Workers organizations, unions, or unorganized co-workers inspired to join might all fill the ranks of a single regiment, or men of various backgrounds from a single neighborhood or city might serve together. This was one of the perceived strengths of the volunteer system, and having men who all knew and respected each other allowed a fairly instant esprit de corps. In any case, men of known qualities, prior military service, or other useful professional or personal skills might be singled out to take an open NCO position.
The short answer then is that your social standing determined a great deal. Officers were generally drawn from the upper classes, and enlistees were generally working class. There were no official rules that enforced this, but it was a robust social element of the war that was largely followed by both sides.
Frederick Lyman Hitchcock's memoirs are terrifically good reading, titled War from the Inside.
Gregory Urwin's The United States Infantry: An Illustrated History has a fairly detailed overview of the dynamics of enlistment and the volunteer system.