Good question. I would say that it's largely to do with the Cold War and the aftermath of the Vietnam-America conflict.

The Vietnamese invasion of Cambodia was actually considered illegal by many of the Western powers. US, China and UK in particular were uncomfortable about Vietnam's alliance with the Soviet Union.

Sadly and shockingly this goes a lot further. A complex agreement between Thailand, China and the West meant that the Khmer Rouge were allowed to keep their power bases and gem riches along the Thai border, in return China agreed not to assist any Maoist uprisings in Thailand, and Thailand secured riches and trade with the big boys.

It may be largely forgotten or brushed under the carpet now, but the US and UK both supported the Khmer Rouge adamantly until the 90s, insisting that the Cambodian UN seat ought to be taken by the Khmer Rouge, supporting, enabling trade and even training their troops. A true low point of foreign policy by both powers. Mostly motivated by a dislike of Vietnam

Edit: A couple of details

The simple answer is that Vietnam did not invade Cambodia because it was killing Cambodians. The primary cause for intervention was not humanitarian. Vietnam knew well about the Democratic Kampuchea (Khmer Rouge) government's policies, just as it knew that the Cambodian government was killing ethnic Vietnamese living in Cambodia, and had made cross border raids into Vietnam to kill Vietnamese. It had more to do with the DK government being allied with China and openltly confronting Vietnam. Please also recall that at this time Vietnam had hundreds of thousands of Vietnamese locked in concentration camps following the end of the Vietnam war in 1975 -- not a sterling humanitarian reputation.

The SRV (the communist Vietnamese government after 1975) was hoping to rid itself of a close China ally on its western border, and to institute its vision of Indochina. As an aside, Vietnamese communist leaders, much like their French colonial adversaries, had a vision of a post-colonial communist Indochinese community that they controlled. The Khmer Rouge threatened that vision. There is a reason why Vietnam stayed the next decade in Cambodia (and recall that the communist regime in Laos was totally subordinate to Hanoi at this time too).

There is of course the angle mentioned above about the tension between the West & China, and the USSR & Vietnam. But the basic precondition of it being an humanitarian intervention was absent. Historians therefore do not credit them.

Goscha, "Vietnam, the Third Indochina War and the Meltdown of Asian Internationalism," The third Indochina War, 2006.