Follow up question: I've heard the Romans and Greeks didn't value the idea of mercy because it went against the idea of justice, which is one of the reasons they hated Christianity. Is there any truth to this?

I'm sure there are people here more well-versed than I am on philosophy. But from the little I do know, the 1st century Stoic philosopher Epictetus wrote a description in his Discourses of the ideal Cynic, who:

must be beaten like an ass, and yet, when beaten, must love those who beat him as the father, as the brother of all.

Epictetus' hero Cynic, Diogenes is also quoted by Plutarch in Moralia as having answered the question "How shall I defend myself against my enemy?" with "By proving yourself good and honourable." Plutarch (also of the 1st century) then goes on to state:

If you wish to distress the man who hates you, do not revile him as lewd, effeminate, licentious, vulgar, or illiberal, but be a man yourself, show self-control, be truthful, and treat with kindness and justice those who have to deal with you. And if you are led into reviling, remove yourself as far as possible from the things for which you revile him. Enter within the portals of your own soul, look about to see if there be any rottenness there, lest some vice lurking somewhere within whisper to you the words of the tragedian: "Wouldst thou heal others, full of sores thyself?"

(Although very similar to Luke 4:23's "Physician, heal thyself!", Plutarch's quote is attributed to Euripedes. And the sentiment of both parallels Luke 6:42's "Remove the beam from your own eye")

They're similar, but not identical sentiments though. I don't think the Greeks here are not advocating the same amount of selflessness that the Sermon on the Mount did. Complete selflessness would be a flaw in your character just as selfishness is - the virtuous path would be to seek complete fairness in ones's dealings. The goal of the Stoic is peace of mind; not to be distressed by other's bad doings - but not necessarily to love them.

I don't think "Love thine enemy" would be entirely alien to someone versed in Cynic or Stoic philosophy. Later stoic philosophers like Marcus Aurelius would be held in high regard by many Christians.

Considering the respect paid to Diogenes despite the many crazy things attributed to him (e.g. advocating public masturbation), I'm not sure Jesus would seem like such a rabble-rouser.

Love thine enemy is a more extreme form of "Do unto others as you would have them do unto you."

Essentially the golden rule, which is a moral standard that has been part of philosophy for as along as there has been philosophy. Most famously is Plato, who, through Socrates, said "...it has been shown that to injure anyone is never just anywhere." The 'Love' aspect may be entirely Jesus, but the sentiment of mercy and non-violence has been around since the Greeks and Taoists at least.