Several factors were at play: first, Gracchus had enormous support from the masses, and the masses wanted the agrarian bill to pass. It had always been understood that a tribune's behavior should always align with the good of the masses (that's his job, after all), and Octavius' veto of the proceedings was plainly contrary to the will of the people in this instance. The sources for this incident aren't great, but Gracchus probably proceeded along those lines: this tribune is not performing his tribunician duties, and should be removed.

Second, the threat of violence. These meetings were raucous, and political violence was becoming dangerously commonplace in the Roman Forum around this time. Once Gracchus maneuvered a vote against Octavius about his suitability to act as a tribune of the plebs, he probably rightly realized that another veto, on the question of the validity of his veto, was a bad idea.

Third, it was unprecedented. There are a few other cases of similar types of things, but in different circumstances. This was probably the first time (that we know about) that a tribune of the plebs was flat out impeached in the assembly. Again, the key here is that it was with the nominal support of "the people' present in the forum that day, including a host of farmers in from the countryside, and it concerned Octavius' ability to deploy a veto which was directly and obviously contrary to the will of the people.