The crucifix became a popular symbol for Jesus Christ and Christianity in general because of its direct and central relationship to the passion and resurrection as well as its connotations of sacrifice and suffering, key themes of the early church. Under the persecution from Nero onwards, crucifixtion was a favourite method of dealing with Christians, a bit of "if you worship him, you might as well die like him" on the behalf of the Romans, something many matyrs, (a common occurance in the early church) were willing and honoured to go through. Comtemporary writers describe how crucifixes were set up along the road to Rome every half-mile and rather than take them down after they died, the were lit on fire at night so that late travellers could easily find the approach to the city.
Other common symbols were/are The Lamb, being both a common symbol of sacrifice in both Judaic and Roman traditions, a symbol for Jesus, drawing on biblical references, the Shepherd is also a symbol of Jesus, being the saviour of man.
The Chi-rho, a combination of the two Greek letters that make up the word Christ, looks like an X over a P, more common in the Orthodox churches today.
The Icthus, or fish, was one of the most common early symbols, coming from the fact that Jesus came from the Galilee Sea region, many of his followers were fishermen, like Simon Peter (which is why the Pope carries a ring known as the fisherman's ring, as the successors to Peter in Catholic tradition) as well as as biblical stories such as the feeding of the five thousand. Early Christians would use this symbol to secretly make themselves known to one another, by carrying some sort of image of a fish.
By Constantine's time, the cross was a common symbol of Christians as can be seen in the legend of Constantine's conversion on the battlefield, seeing a flaming cross in the sky and hearing a voice in the sky telling him that he will win the throne of the Empire if he becomes Christian. So he tells his men to paint crucifixes over the crossed thunderbolts of Jupiter on their shields, gets baptised after he wins the battle, and goes on to be crowned as Imperator, making Christianity the official religion of the Empire.