Caesar and Servilia were indeed lovers. Suetonius claims that he "loved her above all others". Caesar had many affairs, but Servilia seems to have been the longest lasting and most passionate. There were persistent rumors that Servilia's son Brutus was actually fathered by Caesar, unlikely as Caesar was only 15 at the time, but telling as to the extent of public knowledge of the affair.

That said, there is no record of the affair ending, or of bad blood between the two. Not that such a thing isn't possible, but no record of it exists. Caesar was away in Gaul for 10 years, then off fighting his civil war for another 4, so they may have been estranged by the time of his dictatorship. However, following the defeat of Pompey, as Caesar disposed of properties belonging to dead or exiled pompeians, he insisted that everything be purchased at full pre-war value (this in contrast to previous conquerers such as Sulla who allowed his friends to buy properties at a fraction of their value), though Caesar did make at least one exception, that being purchaces made by Servilia.

While it is not known how she felt about Caesar in his last years, or if she had any involvement/advance knowledge of the assassiation plot, she was involved in the councils held by the conspirators following Caesar's death and her advice was highly valued.

(Worth pointing out, the Rome series, while great, leaves out/changes/invents a great deal about the actual historical figures. In truth, Antony was married at least twice before his marriage to Octavia, the second marriage taking place during much of the time covered by the series. Atia also was married and there is no record of any discord between her and Servilia. Much of Atia's character is based on a woman named Clodia Pulchra, a rather scandalous woman of the time, as well as Fulvia, Antony's wife and a major promoter of his political career. Octavian was in Greece with Caesar's army at the time of the assasination. Cato was only 49 when he died, and he didn't go quick. I could go on, but you get the idea.)

One interesting and ironic tidbit about Servilia that the series never mentions is that Cato, Caesar's nemesis, was in fact Servilia's half brother (they had the same mother). According to Plutarch, a particularly embarrasing moment occurred for Cato during debate over what to do with the Catilinarian conspirators: Caesar was handed a tablet to read while Cato was speaking, and Cato, assuming some secret political message being passed, demanded that it be read aloud to the senate. It was, in Plutarch's words "an unchaste letter from his sister Servilia to Caesar". Brutus's wife Portia was Cato's daughter and his marriage to her, following his pardon by Caesar, was a somewhat provocative political statement about where his ideals lay.

Sources:

Adrian Goldsworthy, “Caesar: Life of a Colossus”

Matthias Gelzer, "Caesar: Politician and Statesman"

(This is my first attempt at an AskHistorians answer and I do hope it meets specifications, though I'll take no offense if it doesn't. I'm sure /u/xenophontheathenian could do a more masterful job)