To some degree, you've answered your own question. It fertilizes the crops. It's buried, so pest impact is minimal -- by way of comparison, modern organic fertilizers, made from blood meal, bone meal, chicken feathers, etc. will all happily attract pests if you leave them on top of the soil, but when they are buried, few pests will dig to go after them, and the ones that will would most likely be happy to dig for corn seeds on their own -- I can offer anecdotally that squirrels and other rodents are happy to dig up plantings without fish, and am confident that this could be sourced without effort.

Poor, thin soils are a perennial complaint in New England, so the idea is that buried fish give nutrients, especially nitrogen, which corn (maize for the European readers) is particularly hungry for. I recall that Philbrick's book (pop-history-ish but a good read and, if I recall, adequately sourced -- someone please correct me if I speak astray) gave a good run-down of agricultural problems in New England soil.

I do recall, interestingly, reading that burying fish for seed starting was a European trick rather than a Native American one, and that Squanto (Tisquantum) may have picked up this trick from his European exposure instead of native farmers; the Pilgrims were overwhelmingly "city folks" rather than trained farmers or fishers, and did not bring European best practices with them. I do not recall whether I read this in Mayflower or in 1491, and admit that I have not explored it further, so someone may have more information on this or even be able to counter that argument.