Octavian rose to power after Caesar's death, so it seems, in retrospect, that Caesar must have intended him to be his successor. But Caesar was very close with Brutus as well, and Brutus was of a leadership age, while Octavion was still young. Was Brutus seen as next in line? Might he have entered that role if he hadn't decided to side with Cassius and murder Caesar?
No, Brutus was not next in line. No one was "next in line" because Caesar hadn't altered the Sullan organization of the state.
What would happen in a normal situation in which a consul died was a snap election for a suffect consul (one to serve out the rest of the year). Marc Antony was an ordinary consul, so the two consuls would just continue to run the government until the end of their year, when they would hand over power to the consuls elect and go take up their provinces as proconsuls. There's no reason this couldn't have happened, and it seems to me from the agreement the Marc Antony made with the assassins (general amnesty, ratification of Caesar's official acts) that they all hoped for some sort of return to the status quo ante.
First, Cicero despised Antony and wrote 12 speeches against him (not all delivered, or at least not delivered in person), and this seems to have moved the general opinion of the senate away from Antony.
Second, Antony had been appointed to Macedonia, and he was unhappy with that. He wanted Cisalpine Gaul, where like Caesar he would be in a position to march on Rome, and so exert some political power in the city despite being away. Problem was, Decimus Brutus was already proconsul of Cisalpine Gaul. So instead of going to Macedonia, Antony took his legions and marched on D. Brutus, whom he besieged in the city of Mutina.
Third, when Octavian heard Caesar was dead, he rushed to Rome to at least in part collect his inheritance. He was the heir to the vast majority of Caesar's wealth, and ultimately the one who would execute Caesar's benefactions to the people. When he got to Rome, Antony who somehow had possession of it, refused to give it over. This legal issue rose to the level of discussion in the senate. In the end Antony did hand it over.
At the same time Octavian was demanding retribution on the assassins for the death of his adoptive father. Pietas was one of Rome's cardinal virtues, and Octavian became very popular for the devotion he showed to his also very popular father. Cicero thought he could use Octavian against Antony, and when the senate declared Antony an enemy of the state, Cicero arranged for Octavian to accompany the consuls Pansa and Hirtius, who were sent with an army to lift Antony's siege.
During the battle of Mutina and connected operations, both sides got it bad, and both Pansa and Hirtius were killed, leaving Octavian in command of the Republican forces. Octavian immediately began a policy of hostility towards the assassins, of which Decimus Brutus was one. The senate put D Brutus in charge of the war against Antony. D Brutus expected Octavian to intercept reinforcements coming to Antony from.... Gaul? But Octavian refused to help the assassin at all.
Octavian and Antony realized they both had a beef with people in the senate (remember, all the official acts of Caesar were ratified? That included assigning offices, many of which went to people who would participate in the assassination. Both M Brutus and Cassius Caesar appointed to governorships of provinces in the east), and decided they'd rather fight those guys than each other.
None of this had to happen. Antony could have called a snap election for a consul suffect, everyone could have respected the amnesty, and the hot mess that was the late republic would have continued. But Antony was ambitious, Cicero hated Antony, and Octavian wanted his inheritance and to avenge his father, and so we get another 15 years of civil conflict.