Were there limitations?, were fossil fuels cheaper? Or was it simply not understood enough to make a electric car?
Well, I'm not an historian, but I'm an electrical engineer, which is probably the correct qualification for answering this.
The answer is "energy density". While it's true that there were crude electric batteries in ancient times, they had nowhere near the energy-or power- density in order to move a vehicle.
Energy density is a measure of how much total energy a battery can store per unit of weight. The range of an electric vehicle is determined by how much total energy it's batteries contain. Power density is a measure of how much of that energy is available per unit of weight per unit of time- so how quickly your electric car can accelerate is dependent on how much power the batteries can output.
Energy density is the main limiting factor in designing an electrical car. We've had the technology to make electric motors that were good enough for a car for a very long time. We even had transportation systems that used electricity, but had to draw it from a power line (and thus only travel along a set track). It's actually an even simpler design than using an internal combustion engine, which has a lot of different moving parts and systems and requires multiple types of fluids. What we haven't had is a battery that is cheap, compact, and light enough to carry enough energy to come even close to what gasoline can store.
Lead-acid batteries (which are the kind that gasoline cars use to start) have very good power density, but they don't have enough energy density to make an electric car practical. Early electrical cars used them, but their ranges were severely limited. They also have issues with cycling- they aren't very good at having their energy deeply drained and recharged on a regular basis; they'll eventually stop being able to hold a charge at all. Running a lead-acid battery down to zero can basically kill it, and even if you can recharge it it will take hours.
Modern electric cars use lithium-ion batteries. They've technically existed since the 1970s, but it took a while before they became good enough to be used as ubiquitously as they are today. Even now, though, they're still heavy and expensive enough that you can't just throw as many as you like on the car.
There's also the issue of distribution. Batteries are too expensive- and replacing them is too complex- to swap them out like you can just put more gas in your tank. Even on a large scale, we don't have any good way to store large amounts of electric power. An electric car is useless unless it has reliable access to charging stations- and charging takes a while, so their owners really need one at their house. A rural farm from the 40s that didn't have electricity hooked up wouldn't have had much use for an electric car.
Building an electrical infrastructure takes a lot of time and investment. Gasoline can just be put on a truck and driven wherever. Electricity requires transmission lines, which can only be so far from where it's generated, and which have to be individually built out to every place they'll serve. Gasoline can be stored in a depot to smooth out spikes in demand. The electrical grid has no such reserve; everything that is generate must be consumed at the exact same moment.
An electric car could have been built way back in 1850; there just wasn't any practical way to power it.