Many emperors came to power violently but I've noticed those that came to power peacefully as the officially designated heir were often not the emperors biological son. Non of julio-claudians were the son of their forebear biologically for example. Going through the list it seems that direct succession became more common as time went by (including the eastern empire). Is there why this was the case originally and why it changed?
This winds up being a combination of necessity and the prized Roman love of adoption.
Out of the five Julio-Claudian Emperors, only two had biological sons to begin with. Tiberius had fully planned and expected on having his son Drusus succeed him to the throne, but Drusus died before his father. (Suetonius claims Drusus was poisoned by his wife, but Suetonius thinks everyone was assassinated).
Claudius is a more interesting case. He did have a natural son, Britannicus, who was a mere 13 years old when Claudius died. Rome had not adjusted quite so much to monarchy that it could handle a regency for a ruler not yet of legal adulthood, and the throne passed to Claudius' wife's son by a previous marriage, Nero. Nero was himself a teenager, but was crucially a few years older than Britannicus and over the age of majority. Claudius' wishes that Nero and Britannicus share the throne were ignored and Nero had Britannicus killed.
Augustus, Caligula, and Nero never had sons to try to pass the throne on to, and Augustus famously spent most of his latter years trying to get the throne to one of his grandchildren but they kept dying tragically young.
I don't think there was any lack of intent. Tiberius fully expected Drusus to be his heir until he died, and Claudius did want Britannicus to rule but died when Britannicus was too young to rule.