Why did people wear shoes with heels so often and over such a span of history? Seems like they'd be an encumbrance rather than helpful.
The simple answer for history is the same one you'll get for asking a woman today- it's fashionable, it makes their legs longer, gives the illusion of being thinner, and makes the clothing look better. To look at the historical perspective of this take the one of the first types of heeled shoes- chopines. They were elevated shoes, not unlike our wedges today. Filled in with cork or wood. These were worn in Europe amongst the higher classes during the 15th and 16th centuries and definitely were an encumbrance. Sometimes assistants were needed by the lady to walk. But, that in itself shows wealth. While not nearly as damaging as the lotus foot, it bears some similarities in it's reasoning. Not needing to be able to do labor and to have the servants necessary to do it for you. They were tall, with long flowing skirts, and elevated above everyone else- rather impressive. It didn't hurt to keep their feet out of the mud and gutters either.
Heeled shoes as we know them are made possible in the 16th century when construction changes to include a separate sole from the uppers- early 16th c. where the sole is the same lightweight leather as the uppers compared to an early 17th c. heeled pair of slippers. At first, they are low and meant to help riders keep from slipping in the stirrup. The popularity of everyday shoes using heels comes from gentry, as most fashion did. Catherine de Medici and Charles XIV, both on the short side, did much to make them fashionable in their times. Interestingly enough, this trend adjusts shoes in another manner with the shoemakers using straight lasts rather than right/left to save on the cost. I can use one last instead of purchasing two and the wood heels can be quickly carved to be the same shape.
Heels fall out of fashion around the turn into the 19th century, becoming smaller and shorter throughout the prior century. They come back again in the 1860s, though this time (at first) are made from stacked leather rather than a hardwood structure. Keep in mind that until the 1940s most heels are below 3". And relatively sturdy when higher in height. The narrow heels of the 1780s and 1790s rarely go over 1.5". They were all very comfortable and very serviceable. Flat shoes or sturdy heeled shoes were always available to those that required them- for their job most often. A female blacksmith will chose a sturdy shoe for work, but may have a nice pair of heels for church or attending dances. You look much more elegant dancing in a light pair of heeled shoes. The biggest thing to remember is that shoes today are often poorly made to fit a improper size range. They're uncomfortable because they don't fit right or move with the foot, unless of course we're talking about those 6"+ stilettos, but those have always been considered foolish.
Some of the earliest known heeled shoes were created for a task that they are actually very helpful for: horseback riding. The earliest depictions of heeled footwear are riding boots from 9th-Century Persia. Heeled boots help hold riders' feet in stirrups and more easily stand while riding, which is why riding and cowboy boots have continued to be heeled, even for men. This type of heel spread to Europe in the 16th-century after the first Persian diplomatic envoys to the region. The European upper-class adapted them for both functional (riding) and fashionable purposes and their popularity subsequently spread.
As the Persian-style heels became more popular, aristocrats, most famously the French court, wore increasingly high, ornate and impractical footwear to indicate status.
High heels for men, along with other ornate and non-functional styles of dress, began to go out of fashion in the 18th century. This was due in part to the spread of Enlightenment thinking and its emphasis on rationality and equality. After the French Revolution, the trend of impractically high heels for women also went out of fashion, although lower heels were still worn for the practical purpose of keeping skirts off the ground as well as for the appearance of daintier feet.
Higher heel heights began to re-emerge in the late 19th century and increased in popularity throughout the 20th century as women began to wear shorter dress styles and materials (rubber, molded plastic) that allowed newer shoe designs became more widely available.
Other precursors to the modern heeled shoe include pattens and chopines. Pattens were a platformed overshoe worn for outdoor use by both men and women beginning in the Middle Ages. Since they were intended to protect shoes and clothing from dirty streets, they were often rather high and continued to be worn by women as outwear up until the 19th century.
Chopines were a women's platform shoe popular during the Renaissance. They were initially worn like a patten, but as their height became symbolically associated with status it became fashionable for upper-class women to wear absurdly high versions.
Edit: since clothing fashions are influenced by so many factors, this is a very general summary. For example, military-influenced dress, including heeled-boots, cycled in and out of fashion during the 19th century.