Although Julius Caesar could speak Greek, "Veni vidi vici" was supposedly (according to both Plutarch and Suetonius) a phrase from a letter Caesar wrote to the senate in Rome after his victory over Pharnaces II of Pontus.

Official letters to the senate would have most likely (as reported) have been written in Latin.

Caesar wrote most of his works in Latin. He was also a highly regarded orator in Latin (according to Cicero).

So, though he may have sometimes spoken Greek, he generally operated in Latin.

Most of the upper class of Rome spoke in Greek a lot of the time

I'm not sure this is true. I mean, I agree that the upper classes were generally educated in Greek, but this does not correlate to them speaking Greek predominantly. Where is this claim coming from?

edit: I'm reading up on the topic as we speak, but I would appreciate any input from others.