Why were soldiers sent "over the top" in WW1?

Well, if you're in a trench and want to attack the enemy, you pretty much have to go over the top. How else are you going to get there?

commanding soldiers to walk directly into machine gun fire from an opposing trench seems like an appalling tactic, why was it so common in WW1?

It wasn't a common tactic, and in fact, I have not come across a single instance of such an order being issued.

It is a fairly common caricature of the war, born of a popular conception of the war which has, in the British case, been born through a diet of endless re-screenings of the Last Episode of Blackadder and endless Owen and Sassoon poetry, become so distorted that it has completely broken free from any moorings in reality.

Really, the generals in WWI were not stupid. They weren't fighting in the trenches because they liked it, or because they lacked imagination or the capacity to adapt to innovation. Quite the opposite in fact, as John Terraine sets out in White Heat, the pace of technological change and tactical innovation in WWI was unprecedented either before or since.

The actual casualty rates of WWI were unremarkable. They were equalled and exceeded before and have been equalled and exceeded regularly since. Even the casualty numbers, in absolute terms, were easily exceeded by the second war.

Its simply a matter of perception, particularly in the English speaking world, that the Great War was somehow unique in its horror and carnage, and that's simply because it was the war in which the English speaking people played the largest role - that of engaging and defeating the main body of the main enemy on the main front of a general war, and therefore paid a price commensurate with that role.

To quote John Terraine in The Smoke and The Fire:

No one questions the Duke of Marlborough's claim to be considered a 'great captain' - despite the fearful losses of Malplaquet. The Duke of Wellington, in Spain and at Waterloo, presided over scenes of awful carnage -bad enough at Waterloo to bring tears to his eyes. Many of Napoleon's battles were sheer blood-baths. General Robert E. Lee, often considered an outstanding military genius, never hesitated to accept heavy casualties in the pursuit of victory. The Soviet marshals of 1939-45 were undismayed by the veritable massacre of their men in the same quest - but nobody hounds their memories as the generals of the Great War have been hounded.

Let's look, for a second at what innovations these 'uncaring and incompetent' generals presided over:

• The war saw the rise and demise of the airship as a strategic bomber, the invention of the incendiary round and the the development of systematic air defences including searchlights and anti-aircraft guns.

• The war started with the largest armies having a handful of fragile aircraft suitable only for reconnaissance and ended with specialized strategic bombers, air superiority fighters, close air support machines, and reconnaissance aircraft equipped with cameras and radios, as well as the first instances of aerial re-supply to keep the momentum of an attack going.

• The creation of the world's first independent airforces

• The introduction of flamethrowers and chemical weapons, as well as the development of effective countermeasures.

• The development of the aircraft carrier, and the first widespread use of submarines for economic warfare.

• The invention of the tank and armoured warfare, and the first widespread use of motorised logistics.

• The implementation of squad automatic weapons. 1914 saw the British army with 2 machine guns per battalion. By 1918 these had been centralised into a Machine Gun Corp to able to provide direct and indirect fire, while platoons had up to two machine guns each.

• The invention of the hand grenade and rifle grenades in the format we now them.

• The invention of the mortar, in pretty much the same shape and form it exists today.

• Electronic and signals warfare, as well as the implementation of electronic countermeasures.

• The development of instantaneous artillery fuses, the ability to fire 'off map' without prior registration of the guns, flash-spotting and sound-ranging to identify and then suppress enemy batteries.

• The use of creeping barrages and box-barrages to suppress the enemy until the advancing infantry were on top of them, and to smash-up counterattacks; and 'Chinese-barrages' to tempt the enemy into exposing himself in the belief an attack was imminent.

• The implementation of all the above into a thoroughly modern doctrine of combined-arms warfare, with the effective administration of some of the largest organisations ever seen in human history up to that point.

Now it is possible that generals were uncaring. If the job description involves sending men to kill and die, maybe being a bit uncaring goes with the territory.

But its very hard to reconcile such a blistering pace of innovation with the charge of incompetence.