There is dispute over whether or not worshipping on Sunday came from the traditional Roman day of worship or if it began because that was the day of the week Jesus rose from the dead. Additionally, Christian art was highly influenced by paganism. Much of the symbolism (such as the halo which was used to indicate a solar deity) comes from pagan sources. In early iconography, Jesus is often portrayed the way Apollo is in pagan art: with long hair and a clean-shaven face. Another example of portrayals of Christ is in the early iconography of the Egyptian church. Jesus will sometimes be given an anubis head because of what Anubis represented to the Egyptian, that is, a deity who intercedes on your behalf when you stand before the judge in the afterlife. The portrayals of Mary with the baby Jesus come from the iconography of Isis worship in the Hellenistic era, and some have suggested that the adoption of a theology about Mary was directly influenced by the worship of Isis. The use of relics has its roots in Germanic traditions. When Germans would swear an oath, they would swear the oath on the bones of a great warrior. The church, to avoid being pillaged in the early middle ages, adopted the practice to keep the Germans at bay. It should be noted that many of these things, particularly the art and iconography come into use only to share the Christian message better as a culture. Below I will list several books that might give you a more complete answer if you are looking for it.

E.R. Dodds, "Pagan and Christian in an Age of Anxiety", Cambridge University Press, 1965. (This is an excellent source by a great scholar. It is well written, and it has the benefit of being written by an unbiased scholar who will give a great deal of insight.) Everett Ferguson, "Backgrounds of Early Christianity", Wm. B. Eerdmans, 2003. (This book is a thick one, giving an almost encyclopedic collection of information on all things early christian and Graeco-Roman culture. You will find a lot of information about what you are looking for in there.) I would also recommend getting a good translation of Augustine's confessions. Augustine was a neoplatonist and he will be able to show how a lot of Greek philosophical language came to be used in theology. I would also recommend the Summa Theologiae of Thomas Acquinas. I suggest the Summa because Acquinas was very influenced by Aristotle and the pagan classics. Whenever he cites the philosopher he is referring to Aristotle. Additionally, I would recommend taking a look through the new testament (English Standard Version or New International Version if you want an English translation, Nestle-Aland Novum Testamentum Graece editio XXVIII or XXVII if you know Koine Greek.) as well to give a fuller picture.

Are you specifically asking about the polytheistic pagan religions? Because most people who argue for these influences on Christianity are looking more at things like neo-Platonism, the mystery religions, or Philonic Judaism. These tended to be pantheistic and more ethical/philosophical than the mythic gods of Olympus.