As a born and raised Catholic, this one always confused me. I remember learning about the origins of Santa Claus and the Christmas Tree. But the only thing I remotely remember about Easter is that egg is supposed to signify Jesus's rebirth. But why is a bunny leaving eggs to children with candy?
In the version you describe, "a bunny hiding eggs", the answer is Pennsylvania sometime in the 18th century. It is a... mutation of the old German Lutheran Easter Hare legend according to which the hare was more like Santa. It judged which children had been good or bad and distributed gifts (not eggs in particular, could be candy, toys etc) accordingly. Where this tradition in turn comes from is murky. Rabbits have been considered fertility symbols in various cultures due to their rapid breeding as have eggs. It may well be a cultural assimilation of various older pagan fertility celebrations that gave rise to this legend.
I dislike wikipedia sources but here is one that is actually really good.