I took a class on historical magic and demonology in college but it was a few years ago so I’ll have to see if I can track down the syllabus and any readings. Off the top of my head, I’d recommend the Testament of Solomon if you haven’t already read it. Pseudo-Dionysius also discusses demons in the Celestial Hierarchy, even though it’s mostly a book about angels. For that matter, I think The Discarded Image by CS Lewis is a good overview of medieval cosmology, even though it oversimplifies things. This is all stuff in the Christian tradition, but like I said, I’ll look for my old syllabus (I keep syllabi for this reason) and see if I can pull up some of the books on Greek and Egyptian demonology I read for that class. I know there was some interesting Byzantine stuff. Watch this space!

edit: OK, I found a few things even though I couldn't find the whole reading list. Here are some of the Byzantine books we read for that class:

Traditions of Belief in Late Byzantine Demonology, Richard P.H. Greenfield

Byzantine Magic, Henry Maguire, ed.

Falnama: The Book of Omens, Massumeh Farhad and Serpic Bagci (more on magic than demonology, I think)

The Occult Sciences in Byzantium, Paul Magdalino and Maria Mavroudi

Might be all I can find for now, but these along with the Testament of Solomon are all pretty solid reading, as I recall. Also, a bonus: rather than reading the source material of Pseudo-Dionysius, I now remember reading for class Dionysius the Areopagite and the Neoplatonist Tradition, by Sarah Klitenic Wear and John Dillon. Not totally relevant, but I do remember the explanations in there of how early Christian cosmology conceived of demons.