Japan stopped building castle because of the Edo Bakufu's 1615 order of "One Castle per Province/Domain" to weaken the other daimyōs militarily and decrease likelihood of another outbreak of war. Most domains were forced to demolish a large amount of their defense structures to prevent incurring the wrath of the Bakufu.

Many of the larger daimyōs or those with ties to the Bakufu got off relatively easy. Kaga domain (the Maeda) and a couple others got away with multiple castles because they ruled multiple provinces. Sendai (the Date) and Satsuma (the Shimazu) got away with multiple castles because only the main castle counted as the other ones were "forts". But it's really because Kaga, Satsuma, and Sendai were the three largest non-Shogunate domains. Just in case anyone thought they could get away with it, Fukushima Masanori lost his entire Hiroshima domain just because he did repair work on his castle without permission.

In any case with the onset of peace, there was no point of trying to challenge the Bakufu's orders.

Gunpowder weapons made castle walls cheaper to knock down than they were to build. c.f. https://www.reddit.com/r/AskHistorians/comments/3too5u/when_and_why_did_people_stop_building_castles/