You have to understand that Rome can be described as a "city-state conquering the world" even if it had not conquered the mediterranean yet, it was still a city-state controlling Italy. In practice that means that a great part of Italy was not under direct roman control but in the hands of "the allies". Those allies were other cities joined with Rome with a treaty. Some willingly, others after roman victories won during the last 150 years. Those allies had to supply men to the roman army and so on. So in defecting to Carthage they renounced to the protection of Rome and to all the help they had to sent to them. But defecting to Carthage is not a really good expression has the Carthagian senate was divided over the war. The barcid clan, the family of Hannibal were the bellicist but as they were away fighting the romans, the carthagian senate, where the influence of the Hannon family was important ( they are sometimes called the anti-barcid because it was larger than the family of Hannon who was simply their leader) had no direct control over them. So "to defect to Hannibal" is more clear. In the end it would avoid that city to be threatened by the army of Hannibal which was the last one still on the field after the destruction of the roman forces at Cannae so it was a very logical move for some of those cities.