Off the top, I'd like to point out that "infatuation" is a strong word, as maid cafes and their associated culture are far from mainstream. They're derivative of and restricted to the otaku subculture which, while highly visible on the internet, is not nearly as prevalent in Japan as people like to believe.

Like feudal households in Europe, Japan's feudal elite had numerous domestic servants to take care of the necessities of life. Merchants, too, would often have servants to deal with things, in addition to the free labor provided by the brutal, almost servant-like apprenticeship system then in place.

I honestly don't think any of that has anything to do with the "Meido" fetish, though. To me, that comes from two sources. One, the Meiji-era fascination with all things western and the conscious desire of the Japanese elite to pattern all parts of their existence on the model of contemporary European nobility. This meant the importation of the particular outfit for their servants, which became an enduring cultural icon (an icon which endures in the West, as well). Two, the the growing obsession with kawaii and moe in postwar anime/otaku culture, a sociological phenomenon that deserves whole books. The icon gets subsumed into the fetish, and a new subculture is born. It also has intersections with a separate fashion movement developing around the same time, patterned on the same late-19th-century historical moment of exchange between Japan and Europe. IMO the whole "meido" thing comes from these sources way more than any domestic tradition of domestic workers.

If you're interested in further reading (and have a MUSE login from work/university) this paper goes over the genesis and growth of the subculture in addition to some pretty heavy philosophy/sociology about what it (and specifically the addition of tails/cat ears to maids) could mean in the context of modern Japanese society and consumerism.