First of all, let's make clear where it didn't come from: that story about English archers is definitively false. I won't repeat it in the name of refusing to propagate bad history!

There is an article by Ira P. Robbins about the middle finger, but this Robbins is a professor of law and justice, not history. His legal writing, I'm sure, is top-notch, but I'm not so sure about some of his history.

He claims that the middle finger is Greek in origin, and that the Romans borrowed it. Looking at his examples in Roman literature, though, they all deal with an obscene gesture of some kind without specifying the middle finger. The best one we have is from Martial (Epigrams 6.70):

Ostendit digitum, sed inpudicum,

Alconti Dasioque Symmachoque.

(He showed the finger, but the lewd one, to Alcon, Dasius, and Symmachus)

This clearly indicates that there was one finger Martial regarded as more obscene than the others. The other examples Robbins cites are less clear. Robbins insists that Caligula would offer his middle finger for subjects to kiss instead of his hand, but the source for this story (Suetonius, 12 Caesars) merely says:

...modo ex aliqua causa agenti gratias osculandam manum offerre formatam commotamque in obscaenum modum.

...and when he had occasion to thank him for anything, he would hold out his hand to kiss, forming and moving it in an obscene fashion.

That could have been the "jerk-off" hand motion for all we know. The last example cited by Robbins involves Augustus exiling a actor: (this is also from Suetonius)

quod spectatorem, a quo exsibilabatur, demonstrasset digito conspicuumque fecisset.

because by pointing at him with his finger he turned all eyes upon a spectator who was hissing him.

Can any Classicists help me out by finding any sources that specify it as the middle finger? I'm curious now.