It's possible, but unlikely.

The primary source for the rumor is Plutarch's Life of Brutus in which he says, "For while he was still a young man, as it seems, Caesar had been intimate with Servilia, who was madly in love with him, and he had some grounds for believing that Brutus, who was born at about the time when her passion was in full blaze, was his own son."

There are several problems with Plutarch's throw away line.

First, Caesar was only 15 when Brutus was born, and likely only 14 when he was conceived. His mother Servilia was 19. We do know that Servilia and Caesar became lovers; she was a long time mistress of his. We don't know when the affair started. It was certainly in happening by 63 BC when a love letter written by Servilia was intercepted and read in the Senate. But by then Brutus would have been 22 years old. Caesar as a man in his late 30s is a very different person than he would have been at 15.

Second, Plutarch was writing this 160 years after the fact. That's close to us writing about a suposed affair of Lincoln's. Plutarch wasn't a contemporary.

Caesar did have a long and very intimate affair with Servilia. He did have a close almost familial relationship with Brutus, in part no doubt due to his relationship with Servilia. So it's an easy supposition to make. But there's no real reason to suppose the affair had started early enough to make Caesar a possible father for Brutus.