One type of cross-class relationship between employer and servant was fairly common: when the employer was male, as with Ethel Parks and Charles Bryant. (Technically, Major Bryant was not her literal employer, but as a convalescing officer he was certainly in a position of power; she was working for the establishment where he was a guest.) We know much less about same-sex relationships at this time, but the Philip Villiers/Thomas Barrow relationship would fall into the same category.

As discussed in this previous answer of mine, these relationships are difficult to categorize. Even in situations where both parties were explicitly consenting, there was a power imbalance.

In most, it was great enough - being compounded not just by the fact that the servant, usually a young woman, was employed by the other party in question, but that she was employed by a person/family of higher social standing and income level - that the notion of free consent becomes hard to pin down. At the most extreme end, it's easy to see the power imbalance between a titled aristocrat with a large fortune, married and middle-aged, and a chambermaid or parlormaid who needs her job to earn her livelihood and can't risk turning him down, but even more moderate situations presented the same problem. A servant who was dismissed without a reference would find it very difficult to get a new job, because she would have no proof that she was experienced and trustworthy, so she was very dependent on her employer's goodwill.

On the other hand, apparently-romantic relationships did occur. While we need to keep in mind the obstacles to what we would consider a truly consenting and equal relationship today, we also need to be careful about writing our current understanding of consent over the experiences of women in the past who may not have perceived their situations as coercive. To a great extent, all women faced a significant imbalance of power when it came to accepting a man's romantic overtures, since marriage meant essentially giving him full control over her assets and her body, and she would have little recourse if he were abusive or controlling. Catherine Steedman's Master and Servant: Love and Labour in the English Industrial Age gets into the issue of dealing with these opposing viewpoints well in her chapter on, well, "Love".

There are no statistics that I'm aware of on seductions of maids by employers, but it happened with some frequency. If they were found to be pregnant, they were generally dismissed without a reference for a future employer, and would quite often face the social stigma of being an unwed mother. A marriage between such disparate individuals was extremely unlikely, but there were a few: Sir Henry Fetherstonhaugh (1754 – 1846) famously proposed to a young dairymaid on his estate in 1825, and they married after she went to Paris for a bit of social polish; John Chaworth Musters, a member of the upper middle class, ran off to Norway with a nurserymaid who worked in his parents' home, and the two married in 1887 after having three children (and before having four more) in order to come home and inherit.

Documented relationships, much less marriages, between high-born young ladies and male servants were much rarer. In the early twentieth century, the marriage of a rich girl and her father's chauffeur, like Sybil and Tom, was specifically used as a stereotype of socially inappropriate marriages to be avoided at all costs! The 1912 film The Darkest Hour started with an elopement like this, which resulted in her being disinherited, the chauffeur deserting her, and her only being accepted back to her father's house when she saves a child's life. As in Downton Abbey, this kind of marriage was seen as an extremely strange and unlikely step that would probably result in a break with the young woman's family even if the chauffeur weren't a political radical.