surely if its war, its war.. we kill, rape, destroy, annihilate etc..
Why in history, notably the English were so gentlemanly in there practice of war....treatment of officers, keeping captives, prisoner exchanges and mutual respect etc..
the Japanese, German, Russian, African would appear less gentlemanly... where did all these rules of war come from?
HURRAY
I get to talk about this! So happy (seriously)!
Let's get into it, then.
What you're talking about specifically is referred to as jus in bello, conduct in war (right in war translated roughly). Jus in bello doctrine has developed greatly, particularly over the past century or so, so I'll help explain some of the reasons for it.
One of my personal favorite just war theorists (just war encompasses, by most counts, the morality in starting, conducting, and ending a war), Michael Walzer, has a section on noncombatant immunity and military necessity you'd probably find interesting. His book, Just and Unjust Wars, is very interesting!
Your question comes primarily from what is called the "realist" school of international relations, which says that morals do not exist in things relating to international conduct. What is right is only what can be forced. This is, indeed, the prominent view amongst most people, but the past century has seen the growth of international law (through what is called the "liberalism" doctrine of international relations) and faith in international institutions like the UN and the EU.
Now that we've got a bit of background, let me give you some examples. The German doctrine of kriegsraison, reason of war (this is from Walzer) is important here. As Walzer puts it:
The doctrine justifies not only whatever is necessary to win the war, but also whatever is necessary to reduce the risks of losing, or simply to reduce losses or the likelihood of losses in the course of war.
However, this doctrine is qualified quite often by moral considerations. The question of noncombatant immunity, which you're asking about, is one point there. To discuss some of the reasons morally speaking that it's qualified, the question of expectations of reciprocity comes up, and how these rules might benefit the nations involved.
A "realist", those who say nations only act in their self-interest, can still accept rules of war. In cases like the Laconia Affair, when where (to sum it up) German submarines torpedoed a British liner (the Laconia) which was carrying 268 British servicemen, their families, and 1,800 Italian prisoners of war. Doenitz, an Admiral of the German U-Boat command at the time, ordered a massive rescue effort upon learning the identities of those in the water. Italian warships were asked to assist, and the rescue effort was still ongoing when Allied planes (unaware of what was going on, presumably) fired on the submarines during the attempt. They did little damage, but Doenitz saw the dangers of this situation and issued the "Laconia order which prohibited helping anyone on a sunken ship.
Why are there rules of war, in cases like this, that dictated that the prisoners should be rescued? Why did Doenitz, prior to the Laconia order, say that the people onboard should be rescued? One argument is that reciprocity was expected. If a British ship did the same to a liner carrying Germans in the same situation, the Germans would like to have expected reciprocity. This served the goals of all involved: had the rescue attempt of the Laconia gone well, the Germans would've freed prisoners of war for their cause and taken prisoners of war of their own (though probably returning the family, because they are of little use).
There's also the domestic implications of it. Propaganda in full swing in wars, even today, is typically focused on how civilians are killed or mistreated. It is a huge catalyst to populaces fighting harder. Can you imagine how much more resolve the British would've been given when they found out 268 British servicemen and their families were left to die by Germans who sunk their ship? The gain for the Germans would've been marginal at best, for a significant cost in the motivations of their enemies.
Another good example of this is the question of hollow point bullets. Their use is prohibited in the Hague Convention of 1899, noted here. Looking at it from a "realist" perspective (again, the costs and benefits of allowing such weapons), there is a big incentive to stop the use of such bullets. Why? Because they detract from the morale of soldiers who see the agonizing deaths of their comrades at the hands of them. This could lead to escalations in warfare's costs, and is generally considered to be poor for a national army's morale. Sure, it might be applied equally to both sides if both use them...which leads me to my next point.
The past century, as I said, has seen things like the Geneva Conventions and numerous tribunals for war crimes. Why? Well, part of it can be seen as a pretext for removing leaders unfavorable to the interests of the powers that start the trials, by using morality and international law to their advantage. However, the more common prevailing view is the growth of a cosmopolitan and liberal ideology. To explain those two terms:
Cosmopolitanism is the belief that all human beings, regardless of the nation-state they belong to, are equal. In that sense, it doesn't matter which side loses more lives: war is a moral wrong that places value on your side's soldiers versus the other side's citizens and soldiers, in almost all cases.
Liberalism is the term used to describe the increased faith in international institutions and law as I said already.
It's been said, especially recently, that the new international order being fostered by the UN has led to increased cooperation and reduced destruction in war (and the change in weapons technology has helped). In many ways, this came as the response to the horrors of the World Wars, according to many of the scholars pushing this view. So, to get back to the core of the question, the rules of war are necessary because they, overall, help preserve lives and reduce suffering. They promote peaceful cooperation by restricting things that might lead to escalation or exacerbation of conflict, and restrict things that might lead to resentment as well.
Now, speaking from a purely moral point of view, the question of noncombatant immunity is important as well. Why do we extend this status to citizens who are not involved in the war/war effort?
Well, in many cases the status wasn't extended throughout history. Think back to Dresden, to London, to Hiroshima, to Tokyo, and that's just WWII; the argument rages even still as to whether or not those acts were justified in terms of targeting noncombatants. On the one hand, there is the argument that it would help demoralize and disrupt the capabilities of the enemy.
While that's true, that didn't stop the British from fighting in WWII. It didn't stop the Japanese when Tokyo was firebombed (though the atomic bombs are another question entirely I've answered before, as to why they were different from things like Dresden firebombings). The results are uncertain.
If one is to believe Walzer, the reason we extend this stems from:
The increased benefits of not having your home population bombed when you have to rebuild your nation post-war.
The increased benefits of not having to face revolt and revolution when the enemy has better bombing capabilities.
The idea that noncombatants are morally protected because they do not pose an immediate threat to you.
This is a succinct summary, but you get the idea.
Personally, I subscribe to the view that the rules of war that we've seen came out of atrocity. Faced with atrocious things we now consider war crimes, nations hoped to ease the transition to peacetime and reduce the instance of war, so they reduced the ability of nations to wage destructive wars. The culmination of this effort (as in, the biggest leap forward, progress hasn't stopped since then though) was the World Wars of the 20th century. Both provided scenes of such destruction amongst both civilian and soldier populations that the nations of the world were pressed towards ensuring they could avoid war in the future. This was part of the inspiration for the League of Nations, and though the process has been flawed since (even the UN remains imperfect by far), there has been a gradual shift in that direction because of the general hope of abolishing war among the major powers.
But again, there are those who subscribe to the view that rules of war will be thrown out the window when a real major power war comes up again (if it does) like some were in WWII. We can't tell for sure, but I have hope!
Sources:
Walzer, Michael. Just and Unjust Wars: A Moral Argument with Historical Illustrations. New York: Basic, 1977. Print.
Bova, Russell. How the World Works: A Brief Survey of International Relations. Boston: Pearson Longman, 2012. Print.
Russett, Bruce M., and John R. Oneal. Triangulating Peace: Democracy, Interdependence, and International Organizations. New York: Norton, 2001. Print.
Can provide more if you're interested in learning more about international relations theory!
For a view on non-Western rules of war, Karl Heider's The Dugum Dani is fascinating. They are a highland New Guinea people. There are many instances of war rules in New Guinea (Margaret Mead wrote about it in the 1930's).
The Dani have all manner of war rules. Most highland tribes in New Guinea and the Philippines do have some rules, but the Dani's rules have evolved so that they and their enemy (there are always just the two groups) agree:
They have a whole host of war-related customs, as well, but those are the rules that are followed in terms of launching attacks and battles. One of the interesting things about the Dani (and their enemy) is that both groups have identical levels of technology and have been that way for centuries (at the time the ethnography was written).
No one knows for sure how they worked out these rules, because in theory the two sides do not speak to each other. Since they are genetically and linguistically closely related, it is possible that they both had shared views on war before the family feud began.
At any rate, I can't think of a single culture I've studied (in anthropology) where there weren't rules of war; naturally groups can get into combat with those who disagree on the rules, but war seems to be one of the more rule-bound bits of culture.
Why in history, notably the English were so gentlemanly in there practice of war....treatment of officers, keeping captives, prisoner exchanges and mutual respect etc..
Is this true in general? To my recollection, most notably at the Battle of Crecy, the French were known for (foolishly or nobly, depending on your perspective) sticking to notions of chivalry and honor wrt how battles were to be fought, whereas the English were much more down-and-dirty pragmatic about it.