Augustus was born as Caius Octavius Cai filius. His 'first name', or praenomen, was Caius, on of the most common Roman first names (there never were that many to choose from in the first place). Octavius was the name of his family, the gens Octavia, a plebeian family that had achieved some reknown. His father was also named Caius Octavius, so he would also have been known as 'son of Caius' (Cai filius). If we believe Sueton, he had the nickname, or cognomen, Thurinus, if we believe Cassius Dio, it was Caepias. Both are apocryphal.

When he got adopted by C. Iulius Caesar, his name became Caius Iulius Caesar (Octavianus). By adoption, he had become part of Caesars family, the gens Iulia, and so took their nomen gentile, and also Caesars cognomen Caesar (though he did use it even before his adoption became official). By custom, he would have Octavianus as a cognomen, since that was how an adopted Roman would usually signify their old family, but Augustus and his supporters never used that version, though Cicero does.

In October/November of 40 BC, after the treaty of Brundisium, his name became Imperator Caesar Divi Filius. Imperator was actually more of a title, signifying the holder of imperium, or supreme command, but with Augustus, it became his praenomen, and would become standard part of the later imperial standard titulature. 'Divi Filius' simply means 'son of the divine one', since C. Iulius Caesar had by now been made into a god, so this was the way to refer to his divine parentage.

On January 16, 27 BC, he got rewarded by the senate with the title Augustus, which he took on as a cognomen. From that point on, his official name was Imperator Caesar Divi Filius Augustus. Tiberius would inherit this title.

The confusion here is understandable, and it's not as clear cut as it seems. Imperator, Caesar and Augustus oscillate between names and titles (if there is a significant difference to make in the first place, at least there is much overlap). All became part of the standard imperial title with time. Every emperor was called or called himself imperator (Tiberius f.e. was hesitant, as with the taking over of the Augustus-title, and was still known as Tiberius Caesar Augustus), but it still retained qualities of a title. The same is true with Caesar, which became a standard part of the name of the Emperor when Claudius and Nero, who weren't members of the gens Iulia took on this cognomen. Later, it became also somewhat of a title for the designated successor, when Galba gave the name to his adoptive son Piso (who never became emperor because of being murdered). It became somwhat of a first step on the way to the title of Augustus, which could also be awarded to a successor while the Emperor still lived, as was the case with Commodus, though they were still minor 'Augusti' without the full power of the original one, which only changed much later with Diocletian when the other Augustus had the full imperial bureaucracy and authority for his part of the empire to work with. In any case, what's important that these names begin to fossilize into titles through custom and inheritance.

So, to return to the question, the 'correct' order would be Caesar Augustus, if talking about the first one.