He is considered the first emperor because during his "reign" (period of sole domination), most of the mechanisms which would define and empower future emperors were set down.

In the early years, after Actium, Octavian continually held the consulship, because this was the only way for living Romans to conceptualize that one person now controlled essentially every Roman soldier in the world. The consulship gave him uninterrupted imperium, which would be a central principle of the emperor. He took direct control of Spain, Gaul, Syria, Cyprus, and Egypt, which was the early evolution of the principle that the emperor was always "governor" of the militarized provinces. This ensured that senators could still govern the quiet provinces, but would have a difficult time getting up to trouble. Another element, which came in 23 BCE, was permanent power of the tribune, the tribunicia potestas. This gave Octavian (and future emperors) permanent veto in the Senate, permanent protection from violence, and the ability to call a meeting of the senate at will. Lastly, probably in 19, he got imperium proconsulare maius "greater command," which allowed him to override any other Roman official with imperium, anywhere in the world. In other words, permanent commander-in-chief. It was these three elements which were at the core of imperial authority thereafter: control over certain provinces and therefore control over the legions there; permanent tribune's power; and permanent ability to override orders in ANY province, not just the ones under direct Imperial control. There were some other, minor, features, like permanent pontifex maximus, but the above three were the core.

I go into a bit more detail about the timeline of development here and in a lower reply there as well.