If you read presidential biographies you'll understand the answer to the question and that's it's different in each case. A good start would be the Lincoln biography A Team of Rivals which talks about how he chose his entire cabinet from people he was against while running for president.

Many are picked for which voters they will bring to the table. For instance LBJ was picked because he would bring in the Southern vote and especially Texas. Texas was won by only 46,000 votes with a population of 10 million. Had Kennedy not won Texas he would not have won the presidency.