The short answer is that it's not very clear from Roman accounts or Roman art what the gestures were. The question is still debatable.

The best scholarly guess (but still debated) is that the gesture to kill was:

• Raise the right hand in the direction of the right ear with fist clenched and thumb extended, and swing the forearm down to the left across the face. This is both a downward thrusting motion, and one which turns the thumb downwards. It would indicate striking the losing gladiator, and the motion would be visible at a distance. It could be repeated over and over again.

The gesture for mercy was:

• Hold the thumb concealed inside the closed fingers and raise the right forearm. It would be a static gesture, in contrast to the waving gesture suggested above.

The two gestures would be visible from a distance, and easily distinguishable. You could look at a crowd and immediately see whether there was predominantly a waving gesture, or a static gesture.

Here is a long and detailed academic paper to start with, but many more papers have been published since. http://penelope.uchicago.edu/Thayer/E/Journals/AJP/13/2/Pollice_Verso*.html