In hindsight it would seem a bit of a disadvantage because not only are the cities getting weaker because of paying tribute but they are making Athens strong. So in the case they want to rebel they have nothing. Why would they not instead use the money to furnish ships and contribute them? At least then if they part ways they take the ships with them.
A fleet in the ancient world is not something that you just decide to build one day if you've got the money and boom now you have a fleet. Ancient warships, particularly in the 5th Century when the vast majority of states were still just figuring out what the deal was with naval warfare, were complicated affairs to maintain. Besides the sheer work and cost of building and equipping the ships they would have to be maintained permanently if you wanted a standing fleet--even a single ship would cost a small fortune in regular maintenance, which is why the Athenian state dropped the cost of maintenance on their trierarchs, allowing the state to pay for only the construction of the hull and its equipment (and usually the crew). Rowers in the ancient world were free men, and it was something of a monumental task to keep them trained and paid. Because free men rowed warships, building a fleet was, to quote the OCD, "therefore, a momentous step for any ancient state." Politically a permanent fleet of any size could have enormous effects--since the majority of Athenian rowers were the unpropertied classes the political balance of the city changed entirely. It's not for no reason that even the Romans scrapped most of their fleet after the Punic Wars, and had little interest in building a new one to combat the pirates--most states, if they even managed to construct and equip a fleet, to say nothing of training and paying the rowers, were unable and unwilling to shoulder the enormous burden of a standing navy