They were indeed worshipped as gods, but not quite in the way we'd expect it in modern times. I'll have to start with the case of Caesar because it's quite different from the case of Augustus, but also important for the latter.
When Caesar was deified after his death there wasn't any curriculum of how the imperial cult worked. He was the first Roman to be deified (although not the first one to recieve cult offerings!). Therefore the cult to Caesar is a lot closer to the real gods' than the subsequent emperors'. On the other hand, Caesar was immensely popular in Rome and deified in Rome, which then lead to the cult being very much centralized to Rome itself and not the full empire. If we were to grade him, he'd probably end up higher in status than the deified emperors of the empire, at the same level as eg. Roma, but again, very local. If you've further interest in Caesar's deification I wrote more about it here.

Now, during Augustus the imperial cult really takes shape. The idea of genius was well established in the Roman mind, but the idea of numen wasn't. Shameless copy-paste from a previous post I made:

What the emperor's genius and numen are is rather hard to explain actually. The easiest way to do it is to compare it with the egyptian b4 and k4 if you're familiar with those. Assuming you aren't these are rough descriptions: Numen: The emperor's life spirit. Only the current living emperor had a numen as far as I've understood it. Meaning no one else than the emperor even had a numen. Genius: Today we like to describe it as the soul. However that's a really easy explanation and isn't quite correct. Soul is a christian term. Everyone had a genius, but what made the emperor special was that he had both a genius and a numen and they were both worshipped differently.

Knowing this is really important for the understanding of the cult, because in reality, your question has to be split in two. Augustus was actually worshipped more or less from the time he became emperor in 27 BCE. And that's where things get really complicated. In the western part of the empire Augustus only allowed things like statues, altars and similar (not temples, those are for gods!) to be raised for his own and his family's genii.

I find in the registers of the Senate that Cerialis Anicius, consul-elect, proposed a motion that a temple should as soon as possible be built at the public expense to the Divine Nero. He implied indeed by this proposal that the prince had transcended all mortal grandeur and deserved the adoration of mankind. Some however interpreted it as an omen of his death, seeing that divine honours are not paid to an emperor till he has ceased to live among men. (Tac. Ann. 15.74)

That's a practice that following emperors recognized and the only possible deviation we have from that practice is a temple to divus Claudius from Britannia that seems to have been raised during his lifetime. The situation was quite different in the eastern empire though. The eastern part consisted of provinces that were very familiar with the idea of cult persona; and to complicate matters further, Simon Price differs between divine and heroic honors, but I won't go further into that discussion here.

What that means in reality though is that while Augustus could forbid the worship of him as a god in the west, it wasn't remotely possible in the east. Instead he reluctantly allowed the worship to happen, provided that it was to him and Roma, not him alone. Worshipping just the genius or the numen in the east was of course the optimal course, but if they were to erect temples and treat him as a living god, then it had to be in conjunction with the worship of Roma - Rome's state diety.

Since Augustus was adopted by Caesar, and Caesar was deified at his death, Augustus logically became the son of a god. It's important to note that he did not become a god (while he lived), but merely a god-to-be. During his life, both his genius and his numen were worshipped (separately), which was handled by the sacerdotes (priests) of the different cult institutions (explained here). After his death he notably really became a god and the material we have proving that is rather abundant. It's unclear what kind of god he actually became though, as it is highly unlikely that he became the same kind of god as Jupiter, Ceres, Mars, etc. However, after his death his cult was handled by flamines, which was the same priesthood that worked with the cult of Caesar as well as Jupiter, Ceres, Mars and so on. Meaning that whatever kind of god he became, it was definately at a high enough level to warrant a dedicated priesthood.

TL;DR-ish: So basically, Caesar was indeed worshipped as a god, what kind of god can be discussed, but a god nonetheless. Augustus was worshipped as a divine ruler in the east, but not in the west; he was however deified after his death in 14 AD and thence worshipped as a god.

Not my field of expertise but I know that Julius Caesar was deified by Second Triumvirate (Lepidus, Mark Antony and Octavian) in 42 BC (after his death).

As far as I know this was more of a political move which technically made Octavian descended from a god and helped bolster public opinion to support the Second Triumvirate against Cassius and Decimus Brutus as the two sides prepared for war.

Someone with more knowledge than I can explain how Caesar's deification was treated for the remainder of the empire's existence.

I read recently that Octavian was reluctant to be worshipped as a god, is there any truth to this?