Okay, so, a while back, I jumped down the rabbit hole that is the real nature of harems/seraglios (whatever they're called) in various societies from Japan to Mughal India to Turkey.
I just want some reading material on harems, specifically; the Shogunal Ooku, the Mughal Zenana, the Chinese imperial harem, and the Safavid's harem, but any reading on polygyny and concubinage in various societies.
There was a similar question on the records and sources of the Ottoman Imperial Harem that got some good responses. /u/yodatsracist and I have some comments on readings that might be of interest to you.
*edit, misstated yodatsracist as yodathatsracist, woops. spelling is hard.
Ooku, The Secret World of the Shogun’s Women by Cecilia Segawa Seigle and Linda H. Chance.
While I definitely recommend it, there are real criticisms to make of it, particularly that it takes a few Meiji-era gossipy memoirs too much as fact. This weakness is particularly evident in the sections dealing with the marriage of shogun Iemochi to the Imperial Princess Kazu-no-miya, and what his official "mother" (wife of the previous shogun) thought about her. Just take the reported gossip with a grain of salt, the way you'd take gossip about the British royal family today.
Segawa-Seigle is a respected scholar, though, and her book on Yoshiwara is really recommended, if you're interested in the cultural history of that pleasure quarter.
I highly recommend Servants of the Dynasty: Palace Women in World History edited by Anne Walthall. It contains a condensed summary of Peirce's work (referenced in u/Gaimar's link) as well as chapters on palace women in the royal courts of the Maya, the Mughal empire, the Aztecs, Qing China, and so on.
Ancient Queens: Archaeological Explorations edited by Sarah Nelson provides a good overview of royal women in ancient societies, though it skirts the topic of harems.
For ancient Egypt, there is a UEE article on harems, and the Gurob Harem Palace Project has a reading list on its website.