In fact, they did. in 1945, the Yokosuka MXY-7 Ohka was introduced to the world. It was that strangest of creatures, a rocket-powered glider.

It would be carried to the battlefield in the belly of a Mitsubishi G4M (allied call name "Betty"). Once in the vicinity of the enemy, the Ohka would be released to drop and glide closer, picking up speed as it did. Then at a time judged optimal, the three rocket motors would be fired either sequentially or all together, for final approach to the target.

It was in very many ways the original anti-shipping missile... just with a pilot instead of a targeting computer. It could reach approximately 600mph in a full-power dive, and carried a 2500lb warhead.

Their terrific speed during the final approach proved to be quite the problem for any defenders on a ship. Guns just weren't designed to track something that quick, and there was much less time available to shoot it down. Then, once it did it, the payload was sufficient to do severe damage to a smaller vessel, and substantial damage to a larger.

Despite all this only one destroyer, the USS Mannert L. Abele, was outright sunk by an Ohka. She was also the first ship to be attacked by the rocket-powered kamikaze. USS Jeffers destroyed the Ohka that attacked it, but close aboard; the detonation of the warhead caused damage that required quite a bit of repair time. Finally, USS Hugh W. Hadley was attacked and credited with 23 planes shot down, but took a Ohka hit that caused so much damage that she was eventually written off, but not before a journey back to the US under tow.

A few smaller ships were lost to Ohkas, but on the whole the weapon would have to be deemed a failure. Their short range meant the "Betty" carrier airplane had to get much too close to the target fleet, meaning many of the Ohkas were never launched as the Betty carrying it were shot down by defending aircraft. The tremendous closing speed also meant that the pilot had to be careful on the controls. If everything went right, it could be a devastating weapon. Its just that such conditions just didn't occur very often.

Edit: A little bit of digging in a reference also pulled out the Nakajima Ki-115 Tsurugi. This plane was designed to be very simple to build... essentially a wooden tube with basic wings... and simple to fly. It was also designed to take any engine available, including those from two decades before 1945. The landing gear was only barely more advanced than what had been used on the Sopwith Camel... it had a shock absorber!... and was to be jettisoned on takeoff to be reused.

As it was, it would have made a very nice flight trainer. Instead it was to carry a single bomb of up to 1800lbs and be crashed into the target. There's no question that it would have been easy meat for American fighters, which is why the operational use would have called for hundreds at a time to saturate the defenses.

It was never flown in combat, the war ending before it got out of testing.