So there aren't enough natural magnets of high enough quality in the world to provide for all the machines we have today, chemical power and natural magnets would give electricity a start, but it couldn't have spread without the manufacture of magnets suitable for use in generators. So how and where did this manufacturing start?
Was there a single factory that at one point made magnets to expand the power grid, or were there many smaller entities that were making magnets at the same time?
I asked this over in /r/askscience and was directed here, just fyi if anyone reads this and thinks it might be better suited over there.
Industrial generators don't use permanent magnets to generate power. They use windings over steel laminations: electromangnets, if you will. Using windings allows regulation of the generator output. Many modern generators optionally use a permanent magnet exciter, a secondary component.
I'd like to expand a bit on /u/avarezg's answer...
The main generators use electromagnets to form the magnetic field around the spinning rotor. This is more efficient than using a permanent magnet because you can design the windings to get exactly the intensity and shape of field that you want, and most of the time you want to run in the most fuel efficient mode that you can.
You're right that this is somewhat of a chicken-and-egg problem: how do you get electricity for the magnets to generate your electricity? The majority of the time you can get some power from the other stations on the power grid to get things started, and then the generator supplies it's own power to keep its magnetic field energized.
Every once in a while there is a calamity where ALL of the stations are offline at once which necessitates what is called a Black Start.
Not all power stations are capable of a black start, but those that are usually have a smaller generator, typically diesel, with permanent magnets or a battery. This generator then provides the excitation current for the magnets in the main generator, as well as pump the fuel, open the sluices, raise the control rods or whatever else the main generator needs. Then this generating station can begin the sequence of bringing the other stations online and restoring power to customers.
Black start capability is an additional expense, so it is not added to all power plants, but regulators ensure that there are enough so that grid power can be restored.