The Khmer Empire was a massive entity while Champa was a loose confederation of small city-states, which had much less land than the Khmer Empire. Why did the Khmers have so much trouble dealing with them?
There was an occasion when Cham forces sacked Angkor and killed the Khmer King Tribhuvanāditya in 1177 after the Battle of Tonle Sap.
While the Khmer Empire was able to occupy Champa from 1203 to 1220, Champa was still let go of.
Why did Champa prove such a heavy thorn in the side of the Khmer Empire despite being smaller in size and population and being less centralized?
Three English-available sources here: Coedes "Indianized States of Southeast Asia", Michael Coe " Angkor, and the Khmer Civilization" and a French professor "The Military of the Khmer Empire based on carvings on Angkor Wat, Bayon and Banteay Chmar". There is a 1960s paper on nation building somewhere on the net.
I. How Mandalas Formed in SEA
First of all, it is useful to understand the concept of a mandala. There is three native tribes in mainland southeast asia. The Khmers, Mons and Chams. The word Nagara meant city in Sanskrit, is synonymous as a kingdom in Khmer.
Ignore the monikers: Khmer empire and think only in Kampuchea/Cambodia, at least for now. There are evidences of forts and settlements since 10000 BCE. History of the Khmers began when they can write during the 1st Century CE. The founding father is an Indian trader and Brahman who married the female ruler of a Khmer nagara. Basically, the state already existed long before. According to Chinese deplomatic records, the nation recieved tributes and ruled over dozens of other states and it was later absorbed by one of its vassal in the north. The official name of the nation is not known, but the capital is known. The Chinese called it Funan and called the vassal Zhenla.
This is where Kampuchea first seen, Zhenla is definitely Kampuchea. However, the King of Kampuchea did not absorbed their overlord because of ambitions, he was getting rid of a usurper to the royal family. He is also descended from Funan' s ruling family. So it is a civil war of the Khmer rather than a foreign invasion. One state can rose above all others as the supreme king but any prince of a prosperious city can challenge the central authority. To rule a king appointed his royal families, officials and military captains over an area and they are in charge collect the taxes, administering justice and levying the troops.
Mandala meant centre and sometimes refered poetically as "mother". The people of area centralize around a local leader, who centralized around stronger leader and continued until the strongest of all is the supreme king. Funan is where centre of power began, and when its king is considered illegitimate, the centre of power became the more legitimate ruler of the nagara of Kampuchea. One king ran it so badly, after his death, it fractured to different centres. The Javanese empire able to invade the south. The centre of power went to Java, while the north, there are multitple. Sidenote: Coedes speculated that the Javanese ruling family also claimed descended from Funan but there is not enough evidence.
One of the prince of Kampuchea, who is a hostage of Javanese empire escaped to the Khmer land. By marriages, great officials and moving the capital five times, he became the centre of power of all the Khmer speaking people and became God-king in a the ceromony to declare that Javanese king had nol longer had any hold in Khmer lands. This is start of the Khmer Empire according the traditional scholarship.
II. Naval Warfare and Mandalas during the Khmer Empire
The Chams does not speak Khmer. They had their own centres of power. The Khmer kings had massive populations, massive lands and constructions plannings. Why would they bother to subjudgated the Chams? Managing their empire and centralizing is hard enough. They only had to deal with the Chams due to pirates.
The Khmers can arguably claimed to be the best hydraulic engineers at the middle-ages. Using hydraulic canals as road networks are recorded by the Chineses in the Funan era while the Chams and Javaneses are sailors. An Arabian source described a how a Javanese king managed to sail and beheaded the Khmer king' s head on a plate. It is likely not Angkor' s king as there is no Khmer record of the events or chaos in the following years. Most likely, an outlying Khmer speaking mandala. Basically, seafaring people like the Chams and Malays would not have a hard time in urban settings of the Khmer empire since they are built for boats the same way streets in the US sprawls are built for cars.
The Khmers had more success keeping the Mon in their centre because of closer language ties, land warfare is easier. The Khmer kings had more to deal with than the Chams much smaller city states. In around 1000 CE, the Javanese, Chola and the Khmer empire (and maybe also Burmese) fought a war that ended the Javanese centre of power at sea for a while. The Khmer king went straight for massively speading Kambujadesa influence on land to the north. (You can play this Age of Empire 2).
This is where empire is a fragile concept despite all the glorification. The Tamils claimed that the Khmer gave tribute to the Chola king and acknowledge him as ruler, though the Khmer recorded it as a diplomatic gift of a golden chariot. Meanwhile, hundreds of years later, a Thai chronicle claimed that the Burmese king sacked Angkor. Of course, none of this happened. The Burmese chronicle claimed that they repulse the Khmer from the Malaysia, except most evidences of the invasion is more Khmer influence in the areas, and Chola became surname for Malays. Sometimes, you hold power over a place, sometimes you don't, but since that place is so far away, no one is going to verify and debunk the king claims. (Sidenote: Everytime, someone give a deplomatic gift to the Chinese, it was marked as tributes from a vassals, even if the vassals are stronger) The reason the Khmer empire is known today is because they written in stone.
Back to your question: the Chams fall in and out of their influence. The military of the Khmer empire like many empires are an international force. Chams, Mons, Vietnameses also served in it. The Chinese recruits heavily on steppe nomads during their wars with Xiongnu. The Roman army is led by "so-called barbarian" generals. The British ruled India with Indian sepoys.
Controlling the centre of power in Angkor trumped everywhere else. The Cham invasion you described is against a usurper king. It was repulsed by another prince who hold another centre of power. The capital city is the king concentrated on.
III. China
This part is mainly my speculation. The Chinese records claimed that Champa used to be head by a Chinese ambassasor. Champa had more relations due to naval trades with the Chinese. The giant siege crossbow, the Khmer put on elephants came from the Cham subjects or defectors who took it from either a Chinese shipwreck or instruction manual. China is the most technological advance nation for most of ancient history, probably because they wrote everything down. With more trade, they may have more access to militray technology and doctrine.
As before and now, China' s influence should be never be dismissed. But since they are pretty far away from SEA, they never really had a direct hand on the affairs, except in Vietnam. As people probably know, the Mongols never conquered Southeast Asia beyond parts of Burma. Coedes also mentioned that the glory days of the Khmers seem to coincide with the chaotic days of the Chinese. He made no hypothesis, but I think some sort indirect influence had to exist. It is not like there are think-tanks in the ancient world who can kept the empire informed on developing foreign affairs. So we probably never know. What we do know is the Chinese is far better than the natives in keeping records.