The Harvard historian Jill Lepore had an interesting article in The New Yorker a few years back going over all the various ways and groups people have tried to lay claim to the Tea Party over the years. She points out:

The dumping of the tea wasn’t always such a big deal. In 1823, its fiftieth anniversary passed without observance. Not so the rest of the semi-centennial....

As the historian Alfred F. Young discovered a few years ago, this business with the tea wasn’t called a “tea party” until 1834 [instead it was called 'the destruction of the tea']. The name made it sound like a political party. In the seventeen-seventies, parties were anathema (“If I could not go to heaven but with a party,” Jefferson wrote, “I would not go there at all”), but in the eighteen-thirties parties ran politics.

It wasn't until a generation had passed and most of the actual participants had died that the Tea Party began to be venerated as one of the launch points for the Revolution. Sam Adams and his followers were viewed as radicals prior to the revolution, and a lot of people thought their protests went to far --- a mob trashed the colonial governor's house in protest of the Stamp Act, for example.