I'm currently reading Julian by Gore Vidal, and I was surprised to notice that although they still call each other Romans, at this point most of the elites are Greek, and the only 'Italians' shown are mostly officers in the military. The Augustus himself has never even been to Rome, and most of the elite seem to look down on Latin.

This is a huge contrast to the last book I read about Augustus Ceasar, where Latins controlled the whole empire, and the Greeks were looked down on, and seen as inferior to Romans.

So my question is at what point did the Greeks take over control of the Empire?