It comes from a runner beginning the race before the starter signal is given. In terms of historical roots, the earliest reference to this type of thing that I know of (although not using the phrase in your post title) is from:

Herodotus, The Histories 8.59

When they were assembled and before Eurybiades had a chance to put forward the reason he had called the generals together, Themistocles spoke at length in accordance with the urgency of his request. While he was speaking, the Corinthian general Adeimantus son of Ocytus said, “Themistocles, at the games those who start before the signal are beaten with rods.” Themistocles said in justification, “Those left behind win no crown.”