Electric motors are actually pretty boss in most of the ways that would matter for powering an aircraft. There are a lot of applications where very powerful electrical motors are used to drive big vehicles around. As you've noted submarines had electric motors, and these were actually frequently adapted from the diesel-electric power plants used on locomotives. There were electric cars in the very early days of motoring, and electric motors have also been used for different marine applications and what not as well.

Batteries on the other hand are absolutely awful when compared to energy to weight offered by gasoline or jet fuel, and this has kept it from being used much with aircraft.

Lighter than air flying machines (both blimps and zeppelins) were popular in the early days of flight in large part because they were not as sensitive to weight when it came to their power plants. The first controllable lighter than air flying machine (La France) was actually powered by an electric motor and battery. Gasoline, diesel, and propane(ish stuff) quickly became the fuels of choice as these vehicles got bigger and more practical. There were also experiments with electric flying machines that received power through a ground tether, this has some really obvious limitations though.

In terms of heavier than air aircraft, development came much later. By the 1970s gliders and ultralight aircraft had become fairly mature technology, and at the same time the modern NiCad battery was perfected. These two technologies allowed for experimental "powered glider" type aircraft. Run time on the motors was measured in minutes, but since the goal was simply to get up to an altitude that would allow for good gliding before the batteries went flat this wasn't so bad.

At the same time solar power was enjoying its first big surge of popularity and there were efforts to develop solar powered aircraft. The first that flew successfully was the Mauro Solar Riser, which used a combination of solar cells and a battery pack and first flew in April of 1979. In 1981 the Solar Challenger (developed from the human powered Gossamer Albatross) flew from Paris to London over the course of 5 hours.

Since then there's been further development of this technology, up to the Swiss made Solar Impulse II which is currently attempting an around the world flight. Much better batteries available in modern times have made electric motors practical for unmanned vehicles for both hobby and "serious" pursuits. The military applications for an unmanned surveillance platform with (essentially) unlimited endurance are obvious.

I hope this was helpful, if you have any questions I'll do my best to answer them.