The Kamikaze attack on the U.S.S. Laffey is a good example of how American ships defended themselves.
If you ever watch footage of Kamikaze attacks, you'll see that the sky is filled with puffs of black clouds and lots and lots of projectiles. The term "lots and lots" sounds amateurish, but it's the very best way to describe it.
The puffs of black clouds are the aftermath of a flak shell being fired from the ships heavier guns. It's basically a round that explodes in air, creating shrapnel that will damage or destoy the enemy air craft or disable the Kamikaze pilot.
At lower altitudes lighter anti aircraft weapons fired rounds at a very high rate, with the same intent, disabling or destroying the aircraft or disabling the pilot. In the above video, you'll see tracer rounds (the glowing projectiles) being fired. Tracers are used to track the path of the rounds being fired. Every round is not a tracer, so in between those tracer rounds are more projectiles.
From the ship, the Americans filled the sky with projectiles and shrapnel. They would then radio a carrier group. If possible the carrier group would then send allied planes to combat the Kamikaze fighters.
The main goal was to disable or destroy the aircraft or disable the pilot as to prevent the aircraft from making contact with the ships. Japanese Zeroes were built light weight and maneuverable. Kamikaze pilots believed that dying was the most honorable thing they could do for the Japanese Empire. The end result is a very determined pilot in a light weight and maneuverable plane which makes it very hard to defend against.
The most common way to defend against Kamikaze pilots was sheer firepower.