• Athletics (a similar use as at the baths)
• Perfume (usually mixed with exotic spices and put in one's hair)
• Cosmetics
• Religious ceremonies and funerary rites
• Various medicinal uses

When consumed, it wasn't so much used for 'cooking' as it was added raw to other dishes

The following poem (Mart. Ep. 3.12) jokes about the different uses of olive oil:

Unguentum, fateor, bonum dedisti conuiuis here, sed nihil scidisti. Res salsa est bene olere et esurire.
qui non cenat et unguitur, Fabulle,
hic uere mihi mortuus uidetur.

I admit you served good unguent to your guests yesterday,
But you carved nothing.
It’s a great joke to smell lovely and to go hungry.
He who does not dine and is anointed, Fabullus,
Seems to me to truly be dead.

Fabullus gave his guests great perfume but neglected to feed them. The joke in the last two lines is that corpses were anointed with olive oil.

Much like wine (both for Romans and today), olive oil or perfume was the kind of thing you could bring to a dinner party as a gift.