Most of the evidence for the early Christian church appears in the late first/early second century. In terms of Roman historians, Josephus mentions Christians in the early 90s in the partly interpolated Testimonium Flavianum:

About this time there lived Jesus, a wise man, if indeed one ought to call him a man. For he was one who performed surprising deeds and was a teacher of such people as accept the truth gladly. He won over many Jews and many of the Greeks. He was the Christ. And when, upon the accusation of the principal men among us, Pilate had condemned him to a cross, those who had first come to love him did not cease. He appeared to them spending a third day restored to life, for the prophets of God had foretold these things and a thousand other marvels about him. And the tribe of the Christians, so called after him, has still to this day not disappeared.

Antiquities 18.3

John Meier's reconstruction in his A Marginal Jew, Vol. 1 strikes me as convincing.

Tacitus, the Roman historian, mentions Christians in Rome under Nero in Annals 15.44

Consequently, to get rid of the report, Nero fastened the guilt and inflicted the most exquisite tortures on a class hated for their abominations, called Christians by the populace. Christus, from whom the name had its origin, suffered the extreme penalty during the reign of Tiberius at the hands of one of our procurators, Pontius Pilatus, and a most mischievous superstition, thus checked for the moment, again broke out not only in Judæa, the first source of the evil, but even in Rome, where all things hideous and shameful from every part of the world find their centre and become popular. Accordingly, an arrest was first made of all who pleaded guilty; then, upon their information, an immense multitude was convicted, not so much of the crime of firing the city, as of hatred against mankind

There's also the Didache, which is more or less a manual of order for Christian communities. It was not preserved in the New Testament and is considered apocryphal. In addition, Ignatius of Antioch wrote a series of epistles in the early second century, as did Polycarp, another early Christian bishop.

1 Clement, which is usually dated to 96, explains that Peter and Paul met martyrdom in Rome.

About Paul, the best book to read is probably Framing Paul: An Epistolary Biography, by Douglas Campbell. An introductory work about the early church is probably more than adequate; I used Joseph Lynch's Early Christianity: A Brief History, though W.H.C. Frend's The Early Church and The Rise of Christianity are probably just as good, but a little older. Interpretations of material change, but generally speaking, we have the same data now as we did 50 years ago.