I'll leave the Roman imperial cult to someone else, but I've provided reading suggestions for ancient Egyptian religion and Assyrian religion.

You may also find the reading suggestions on Sumerian society (most of the suggestions contain chapters on temples and religion) and Sumerian mythology helpful.

Religions of the Ancient World: A Guide edited by Sarah Iles Johnston covers the religion of Egypt, Mesopotamia, Syria and Canaan, Israel, the Hittites, Iran, the Minoans and Mycenaeans, Greece, Etruria, Rome, and Early Christianity. It's a pretty useful resource.