He was born with the name of Gaius Octavius. After he was adopted in 44BC he adopted Caesars name ; Gaius Julius Caesar Octavianus. (Note that Octavius gained a -an, due to roman naming standerds relating adoption.) After this he was mostly called Caesar by Romans.

In 27BC, after his victory of Mark Anthony and Cleopatra the senate voted to give him a new title, Augustus. His full name at that time was Imperator Caesar Divi Filius Augustus. After this he was mostly referred to as Augustus. ( The Imperator is a military title, Divi Filius is a title marking him as 'The son of god/divine')

Sources : Galingsky, K. Augustus, Introduction to the life of an emperor. Cambrdige, 2012 edition.

Noble, Strauss, Western Civilization, beyond Boundaries. Wadsworth publishing, 6th edition.

In his letters to Atticus, Cicero used the name Gaius Caesar to refer to Octavian after his adoption by (Gaius) Julius Caesar.

After Octavian was granted honours (effectively became emperor) his official name was Imperator Caesar Augustus, but he was referred to as Caesar Augustus, Augustus Caesar (by Virgil, perhaps for metric reasons), or just Augustus (by Tacitus).

All information and examples from Anthony Everitt's The First Emperor: Caesar Augustus and the Triumph of Rome (2006).