There are many motives behind Vietnam but it appears that the main US interest was to not lose another country to communism. If this was the case then why did they oppose the Geneva Accords of 1954?
"We shall seek to achieve unity through free elections supervised by the United Nations to insure that they are conducted fairly."
-Beddel Smith US delegate
They knew that the majority of the country favored communism and the installed French state was incredibly unpopular.
How is it that the US was so successful in subverting the media in believing that they were defending Vietnam when they were playing both sides of it? They called for free elections and then they oppose the majority rule in Vietnam.
This question is difficult to answer because you've asked it while strongly asserting several things that are not correct.
They knew that the majority of the country favored communism
The majority of people did not know what communism was. This was a peasant society.
the installed French state was incredibly unpopular.
The State of Vietnam (Etat du Viêt-Nam/Quốc gia Việt Nam) government certainly was contested by the Democratic Republic of Vietnam (Viet Minh), but it had support. More Vietnamese died fighting for it, much more than the number of Frenchmen. Refugees continued to flood out of the Viet Minh held areas and into urban areas controlled by the State of Vietnam and French. This government's goal was not to defeat the common Viet Minh foot soldier, it was to struggle for independence through political, peaceful means. Like the DRV, their goal was independence. They believed, if they could do this, they would win people over to their government. They did do this successfully, though not enough. Beside refugees, plenty of Viet Minh officials abandoned the DRV when it was clear that the government was dominated by the communist party and that they were intent on following Chinese and Soviet models of development.
The State of Vietnam government was the legal government recognized by Washington. It controlled about half of the country's population and the cities from Hanoi down to Saigon. Form the State of Vietnam's perspective, they did not want to forfeit half of the country and its people to the DRV government. The State of Vietnam, soon led by Ngo Dinh Diem, opposed the accords, and of course, the referendum as well. The Eisenhower administration, which had supported the State of Vietnam since 1950, supported that position. They did not want to place the fate of Vietnam in a simple referendum in 1956, nor abandon their non-communist ally Ngo Dinh Diem.
If you're interested in books or articles on the subject, I would be happy to suggest some.