While there have been rumors of Philip having affairs for years, there has never been any concrete proof. Sarah Bradford, who wrote Elizabeth II: Her Life in Our Times (2011) is the only biographer who claims to have confirmed affairs. Yet a rival biographer, Gyles Brandreth (Philip & Elizabeth: Portrait of a Marriage) investigated Bradford's claims and was unable to get confirmation. The women Bradford linked to Philip - Katie Boyle, Sacha Abercorn, and Penny Brabourne - all denied to Brandreth ever having slept with Phillip, though they did confess friendships and flirtations with him.

Bradford also repeated the widespread allegations that Pat Kirkwood, a British actress, had a long-term affair with Philip, though Kirkwood denied it on many occasions. She did, however, receive letters from Philip, which she bequeathed be given to Philip's official biographer and only published after both their deaths (Kirkwood died in 2007).

The only time the Queen ever publicly commented on the speculation was in 1957. In 1956 Philip went to sea on the HMS Britannia and was still there when, in early 1957, the Baltimore Sun alleged Philip was having an affair with an unnamed woman he'd met through a photographer friend. The allegations caused enough of an uproar in the press both in the U.S. and in the U.K. that the Queen's office released the statement: "It is quite untrue that there is any rift between the queen and the Duke.” The Queen then traveled to Portugal to publicly reunite with Philip, which squashed the rumors for the time being.

So until a woman comes forward with a public confirmation of an affair with Philip, the rumors of him being unfaithful remain only allegations.

It is well known that the two have a good relationship now. How did his opinion of her change?

As a follow-up, Was there rumor of Philip being gay? The Crown is really making it seem that way with his portrayal