They spoke Latin. Education in Greek was common, especially among elites, and there were immigrant enclaves in Rome, who are richly attested by epigraphic evidence. But these are obviously distinct things: you aren't asking about immigrants.

99.9% of all communication between Roman elites that we have access to was in Latin. The exceptions are sparse. The emperors Claudius and Marcus Aurelius wrote books in Greek; but Suetonius' testimony about Claudius' skills in spoken Greek (Life of Claudius 42) indicates that his conversational skills were limited to a limited range of catchphrases -- and perhaps even more importantly, that his using Greek at all was something notable. Suetonius also tells us that Augustus never got comfortable enough in Greek to hold a conversation (Life of Augustus 87). Tiberius spoke Greek very well, but as a matter of policy insisted on the use of Latin (Life of Tiberius 71). Marius knew no Greek at all, Plutarch tells us (Life of Marius 2.2).

Moving away from emperors and rulers, you may perhaps have seen it pointed out that Quintilian recommends a child's education begin in Greek straight away (Inst. or. 1.1.12). This certainly attests to the importance of Greek in elite Roman education. However, two things. First, there's an obvious reason for this: children learn the language of their environment anyway, so there's no need to train them in the speaking of that language. Quintilian's evidence actually shows that their home language was not Greek. Secondly, notice which language Quintilian himself is writing in: it's Latin.

Technical manuals that we have from Rome are all in Latin (e.g. the agrimensores), as are treatises by medical writers (e.g. Celsus), grammarians and antiquarians (Aelius Stilo, Varro, etc etc) and astronomers (Hyginus). When Fronto writes letters to the imperial family, and when Cicero and Pliny write letters to their elite friends, they invariably write in Latin. Cicero's friend Atticus was a notorious hellenophile (hence the name) and lived in Greece, and even wrote books in Greek, but their letters were invariably in Latin. Cicero rarely uses a word or two of Greek with Atticus, but that's a bit of camaraderie chez nous, not the main language of discourse.

There's no case to be made that Greek was a preferred language among the elite. A more international language, yes: a handful of authors did write in Greek, because that was the language for communicating information to the entirety of the eastern Mediterranean. But for domestic communication, Latin was the only option.

Some bits of testimony are prone to being misused, like the Quintilian passage I mentioned above. Another is Juvenal, Satires 6.185-196:

... what's more revolting than a woman thinking she isn't
beautiful unless she turns from Tuscan [i.e. Latin] into Greek?
From pure Sulmonian into Cecropian? Everything in Greek!
Their shrieks are in that language, their anger, their joys and worries,
all the secrets of their soul that they pour out. But wait, there's more!
They go to bed in Greek. Now, for girls, you might allow that.
But are you really still going to be Greeking when your 86th year
is knocking on the door? That language is obscene
in a little old lady. Every time you interrupt yourself with a lewd
[Greek] 'life and soul!', you're taking things that should only be spoken
under the covers and using them in public.

As with Quintilian, don't just look at what he says, look at why he says it. (1) The people Juvenal is criticising here are expressly not elites: he's looking down on them scornfully. (2) The idea he gives of Greek as the language of love doesn't mean Romans actually spoke Greek in private, any more than the cartoon character Pepé Lepew shows that 1950s Americans spoke French in the bedroom.

More details and references here in a piece I wrote five years ago.

Edit: Fronto, not Frontinus. Oops!