In ancient Spartan society, the Ephors (city councilmen) would declare war every autumn on the Helots (serf/slaves) as a means of population control. During this time any Spartan citizen could kill a Helot without repercussions.

Helots outnumbered Spartans, by some accounts, 7-to-1. The formation of various - possibly clandestine - organizations like the Crypteia, were done out of fear of the numbers of the Helots. There are other behaviors, like Spartan citizens (Equals) never going weaponless, and locking themselves in their homes for fear of being disarmed or an uprising.

However, there are some accounts of Helots attending Spartans on various war campaigns; something unlikely to happen if they feared rebellion (edit: or abandonment). And other sources claim Helots were more serf-like than slave-like. They were less likely to be sold then other Greek slaves, allowing them to form families. Indeed, it is speculated that the sale of Helots would have been far more effective population control than annual slaughter since their populations continued to remain stable and grow while other slave populations in Greece were more tenuous. They were also allowed to own property and purchase their freedom; a status that elevated them above the average Greek slave.

On the other hand, there's an account by Myron of Priene (a biased account), that all Helots must submit to a required amount number of beating*(s)* annually to remind them of their status as slaves.

Edited to add: There were several revolts and rebellions by the helots in the fifth century B.C., and yet they attended the Spartans on a war campaign in Plataea fifteen years prior. Forty years after that slave revolt, 20,000 slaves fled Athens to take refuge with the Sparta army in Attica. Needless to say, the relationship between the Spartans and Helots was complex.

Your question is a bit oddly worded. Waging war is typically thought as an action one nation-state commits to another nation-state. Could you clarify?

Do you mean a ruling party using military force to control the population of a group outside its borders? It would be difficult to argue that population control was ever the chief motivator of war. The American Indian Wars is the closest example that comes to mind. Although that was motivated more so by territorial gains and the ideals of Manifest Destiny rather than population based arguments. Other examples would include Third Punic War and the Qing campaigns against the Dzungar people in the 18th century.

There were certainly conflicts that were fought because one nation or group wanted to limit the expansion of another (e.g. the Crimean War). You could argue that increasing population would be a natural progression from territorial expansion, but it was rarely explicitly stated as such. That wasn't always the case, with the Holocaust being the most significant antithesis to this idea.

But I think you meant more towards a ruling party using military force to control a population within its borders. Looking at western Europe, Thomas Malthus's work served as a watershed for the idea of population control. He essentially argued for (what would later be called) carrying capacity, where populations, human and animal alike, would eventually collapse when they grew too large. There is a controversial argument that this concern contributed to the British government's ineffective response to the Irish Famine of 1845-1852. Farther east, the Holodomor (1932-33) is contentiously considered to have targeted Ukrainians with artificial famine. Neither case really represents direct military intervention; both governments instead relied on the military or a militarized police force for policy enforcement.

On a similar note, you can also look at national sterilization policies. For example, the US's sterilization program in Puerto Rico in the mid-20th century. Some argue that it was a designed program of population control. Others claim that the sterilization was willingly pursued by Puerto Rican women, but that this decision was heavily influenced by a limited access to other forms of contraception and ignorance of the procedure's permanent nature.

If you're talking direct military action, would you consider campaigns of internal genocide population control? The Ottomans used military force to eliminate the local Armenian population as well as the Assyrians and Greeks. Military forces in Rwanda greatly reduced its Tutsi population. I could give more examples, but these events aren't typically considered to be motivated by a desire for culling or some other form of population control.

I'd say the modern public understanding of overpopulation concerns didn't really pop up until the book The Population Bomb came out in 1968. Otherwise, such lines of thinking tended to be contained within the eugenics movements of the 19th and 20th centuries.