Pol Pot's revolution was an entirely peasant based uprising. Democratic Kampuchea was an attempt to turn the entirety of Cambodia into a peasant based agrarian society. The Cambodian peasantry didn't know private property. All land and property belonged to the French, the Aristocracy, and the Japanese. The Cambodian peasantry didn't know education. The French purposefully kept literacy and education rates as low as they could to prevent political unrest. Once the Japanese were run out of Cambodia, a new Aristocracy took control and through Prince Shinouk's attempts at neutrality Cambodia began to be infiltrated with Vietnamese. Cambodian peasant relations with the Vietnamese had always been sour as the Vietnamese were better educated and enjoyed higher privileges in Cambodian society than even Cambodian peasants. Eventually a military dictatorship came to power, one that allied with the United States and was complacent in the death of hundreds of thousands of Cambodian innocents. The peasants who had never earned much money were packed into cities to escape bombing runs, left with no money to buy food or provisions, left to starve.

According to the Khmer itself, the US bombing raid was the final rallying cry for the Cambodian peasantry, which fully allied itself with the Angkar, the Khmer Rouge before they fully adopted Marxist/Communist posturing.

The Khmer state reflected what the Cambodian peasantry knew: It was an agricultural state, one that favored agricultural labor over education, one that favored full communalization of property, one that favored Cambodian ethnicity, one that favored traditional Cambodian peasant life. Property, education, money, multi culturalism were all symptoms of things that had kept the Cambodian peasants poor and destitute, and in the new Cambodian state, those traits would be eradicated.

Pol Pot was overwhelmingly popular with the Cambodian peasantry. Enough to empty entire cities through force.

You know what happened to the Cambodian people afterwards and Pol Pot eventually began to alienate the peasantry through over work and dehumanization and using starvation as a weapon and means of control ("If someone is very hungry, the Angkar will take him where he will be stuffed with food."). When the Khmer Rouge were removed from power through collaborators with the Vietnamese a sizable portion of the Khmer Rouge forces went with Pol Pot into exile. These forces were the ones that came from the provinces within Cambodia that saw the least amount of strife, as certain districts did not suffer nearly as immensely as others. They were also propped up with US and UK funding but their entire goal was to remove Vietnam from Cambodian soil.

In interviews post downfall Pol Pot was steadfast in believing that what he was doing was right, and he still had support from a faction of the peasantry. While I've never heard that particular saying in your question, Pol Pot's isolation from the absolute worst situations in the Khmer Rouge, his belief that all that oppose the Angkar were not true Cambodian peasants, and his continued support in exile gave him all the vindication that he needed.

Sources:

The Khmer Rouge: Ideology, Militarism, and the Revolution That Consumed a Generation

Pol Pot's Little Red Book

Brother Number One: A Political Biography Of Pol Pot

Pol Pot Plans the Future:

Nate Thayer's AMA and the reference interview as well.