Pretty much every U.S. President has either been a Vice President, Cabinet Member, Senator, Governor, or high ranking military figure. Abraham Lincoln & James Garfield are the only two Presidents who held a lower office (Both members of the House of Representatives. Garfield was a member of Congress for 18 years up until he became President. Abraham Lincoln was only in the House for two years and he left 12 years before he became President. He was a pretty prominent lawyer in Illinois.

Edit: A lot of Presidents can easily be described as underdog types. Obama, Clinton, Nixon, and many others didn't grow up particularly well off.