Most Wikipedia (I know, I have sinned) articles regarding Jesus and his historicity say that it is a well accepted fact among historians that a historical Jesus existed.
However, I have only found the usual 2 sources relied upon which are Josephus and Tacitus.
The issue is that neither Josephus nor Tacitus were alive when Jesus was supposedly alive.
Moreover, I have read conjecture that Josephus' account in the "Antiquities of The Jews" was actually a much later forgery by Christian historians.
Finally, is the Bible historically accurate and can we rely upon it if there is no other extra-biblical reference?
This matter of a historical Jesus has ALWAYS confused me and I have been left with only more and more questions every time I touch the subject. Throughout my schooling though I was curious about the matter and was always told things like
"There's more evidence for Jesus than for Caesar" "Jesus existed, scholars agree on that, of course they disagree on his godliness"
But the strange thing is that I went to a non-denominatinal Grammar school which was strictly unaffiliated with any religion. So I took these assertions to have more weight as the people telling me weren't devout Christians.
I don't have much more to add than to refer to my answer that is in the faq, but let me address a few points:
Yes, Josephus and Tacitus are the two major sources outside the Bible that are referred to. No, they weren't contemporaneous with Jesus. But very few ancient historians write about contemporary events, they generally write about earlier events. In the case of Josephus, though, he is born 37 AD and writing about near-contemporary events.
These two sources aren't really direct sources that support the existence of Jesus, they are indirect sources that attest to the existence of Christians who attest Jesus' existence. This is still indirect evidence.
The Bible is a collection of documents, not a singular 'entity'. Each element is available for historical examination. It's poor historiography to exclude the Bible as evidence because it's "religious" or "biased" - if we excluded documents on that basis we'd never do history at all. The method to employ is to investigate biblical and non-biblical documents historically and academically.
"more evidence for Jesus than for Caesar" is usually an argument decontextualised. There's more (in a numerical sense) and earlier textual attestation of biblical documents than Caesar's writings. That's important, but it's not the whole story.
We have a FAQ section covering this. There's only a few answers listed there, if you search the sub you'll find many more.
Also worth mentioning are two things. Last year, when world renown Early Christian historian Bart Ehrman (whom is Agnostic) was given a lifetime achievement award from the Freedom from Religion foundation, [he was asked during a Q&A why he's so certain that Jesus existed when she (the person asking the question) stated that she sees no evidence at all.] (https://www.youtube.com/watch?v=o4q3WlM9rCI) This is the best part of his response:
This is not even an issue for scholars of antiquity. There is no scholar in any college or university in the Western World who teaches Classics, Ancient History, New Testament, Early Christianity, or any related field who doubts that Jesus really existed. Now that is not evidence, just because everybody thinks so doesn't make it evidence. But if you want to know about the theory of evolution verses the theory of creationism, and every scholar in every reputable institution believes in evolution, it may not be evidence, but if you have a different opinion, you better have a pretty good piece of evidence... You want to go where the evidence goes. I think that Atheists have done themselves a disservice by jumping on the bandwagon that Jesus didn't exist because frankly it makes you look foolish to the outside world... You are much better going with the historical evidence and arguing historically."
The other thing worth mentioning are some great and well-respected books on this topic. The two great books that I always recommend on this topic are below:
I personally recommend the Ehrman book because he's an engaging author and is pretty objective, since he himself is an Agnostic.