Jesus was a Jew, but he started a movement that became something else entirely. Muhammad similarly started a "new" religion, but what was his "original" religion? Was he a Jew? Was he a Christian? To my understanding, these religious traditions existed in his milieu, but the relationship is unclear to me.
I know that Islam claims to be the ideological descendant of Abraham's pure religion through Ishmael, but that seems rather vague. Is there a textual tradition or remnants of an oral tradition that describes why the Israelite branch eventually became apostate?
I also understand that Islam recognizes Jesus as a great prophet, but why? Surely by Muhammad's time the Christian tradition was well established, and Jesus was firmly believed to be god himself come to earth as the Jewish messiah. What tradition of Jesus did Mohamed adhere to, and what was the basis for that tradition?
I know there's several questions here, but I guess the overall question could be rephrased as, what "scriptures" did Muhammad and his contemporaries hold sacred that provided the backdrop for why people started following him and his message?
Pre-Islamic Arabia was relatively diverse. It was near major trade routes at the time and an intersection of several distinct religious traditions: Zoroastrianism in Persia, Christianity and Judaism in Africa (modern day Ethiopia) and the Levant, and Arabian polytheism.
Growing up, the Prophet Muhammad was an orphan who traveled as a trader with his uncle. He would have been very metropolitan, being exposed to a wide variety of religious traditions and stories. In fact, there is a tradition saying met a Christian monk (named Bahira) when he was a child, and the monk predicted that Muhammad would be a prophet. Also, Muhammad's first wife (Khadijah) had a close cousin who was a Christian convert. So Muhammad was definitely familiar with Christianity.
Growing up, traditions/biographies of Muhammad say that he was a spiritual person, but do not say that he was devoted to a particular religious tradition. Muhammad's tribe (the Quraysh) where the guardians of the Kaaba and traders during most of Muhammad's life. As Muhammad was an orphan, he likely didn't rock the boat on that too much. Still, Islamic tradition is that Muhammad was not a polytheist (although that could be biased, for obvious reasons). At the time, the Kaaba was a center of polytheistic worship and trade. As protectors of the Kaaba, the Quraysh made lots of money by being accepting to different pilgrims that would come by, and there was likely no strict dogma of their polytheism--because excluding worshippers would cost them money (and also because polytheism is generally welcoming to worship of new deities).
A few things to note about Jahilliya (Pre-Islamic) religion:
Most religious traditions did not have the same kinds of strict/unified dogma that we expect of them today. A Christian in Aksum would likely have very different ideas about Jesus than a Christian in Rome or Jerusalem. Most agreed on the general stories, but there was a lot more intermixing between religious traditions than we might understand now.
While religious beliefs might be a bit of a melting pot, religious affiliations could be more strict, in that it was expected you would practice (at least publicly) the religious traditions of your tribe. One reason for this is that religion reinforced the power structure. For example, in Mecca the Quraysh derived economic power from being the protectors of the Kaaba (a polytheistic temple), so they naturally were upset when Muhammad started criticizing polytheism, since it threatened their position.
It might be interesting to note that Christianity and Judaism were influenced by Zoroastrianism, and Islam was maybe even more influenced by Zoroastrianism than any of the Abrahamic religions. Zoroastrianism is a bit of a unique worldview, in that it is has some "dualistic" traits. Zoroastrians believe that there are two forces (good/bad or order/chaos)--although they also seem to believe those forces emanate from a single divine force. While polytheists see gods as relatively neutral, and monotheists see God as a single entity, Zoroastrians see two opposite forces of relatively similar strength. This might be one reason why Iblis (the Devil) plays a larger role in Islam than we see in Jewish/Christian traditions (historically, at least).
Finally, after Muhammad received his revelations, he was threated by the Quraysh and had to leave Mecca. Notably. he received shelter and protection from a Christian King in Aksum. Tradition is also that the residents of Medina (where Muhammad and the early Muslin converts stayed) was home to Jewish tribes. When Muhammad was asked to lead the community, he drew up a constitution that recognized the rights of all residents (Jews and Muslims), which might be informed by his life growing up around so many religious traditions.
SUMMARY
Basically, to answer your question, it is fair to say that Muhammad was familiar with a wide range of religious traditions, including many of the Christian/Jewish stories contained in the Qur'an. These, along with Arabian Polytheism and Zoroastrian were likely influences on both him and his followers.
I might push back a bit on your statement that "Surely by Muhammad's time the Christian tradition was well established, and Jesus was firmly believed to be god himself come to earth as the Jewish messiah." While the Roman churches came to some agreement on this in the 6th century, there was nothing like the "Council of Chalcedon" for the Ethiopian/Coptic/Syrian churches that Muhammad would have been most familiar with. It wouldn't be far-fetched if an outsider actually viewed early Islam as a Christian sect (rather than an entirely new religion). I think that the distinctions between Judaism/Christian/Islamic theology--at least in the early 7th century--would not be as stark as some see them today.
More can always be said, but this older answer, as well as this one, both from /u/frogbrooks, look at pre-Islamic society in the region so should be of interest for you.
In Islamic tradition, the Prophet Muhammad was never an idol-worshipper or a polytheist. He belonged to a small, informal group known as the hunafa (the renouncers). They rejected polytheism and idol worship, and instead adhered to Abrahamic monotheism.
They don't appear to have had any established hierarchy, rituals or practice (and if they did exist, they are lost to history), but Muhammad himself was fond of spiritual retreat. He would depart the city of Mecca to spend time in the solitude of the nearby cave of Hira, for contemplation and meditation. It was during one of these retreats that it is reported that the angel Gabriel came to Muhammad and revealed the first verses of the Qur'an: "Read, in the name of thy Lord who created. [Who] created mankind from a clot of blood. Read, in the name of thy Lord who taught by the pen. [Who] taught mankind what he did not know."
Judaism and Christianity also existed in Arabia at the time of Muhammad, but the primary religion in the Hijaz region of Arabia was overwhelmingly Arabian polytheism. Jewish tribes centred around the city of Yathrib (later called Medina), and played a larger part in the second half of early Islamic history after the Hijra (the departure of the Islamic community from Mecca to Yathrib). Christians were frequently encountered on northern trade routes to Syria, but the most significant contact occurred prior to the Hijra, when a large section of the Muslim community fled to the Kingdom of Axum in modern-day Ethiopia, as a result of the violent persecution they were facing at the hands of the Meccan polytheists. Muhammad himself remained in Mecca. In Muslim tradition, that section of the Muslim community was pursued by the Meccan polytheists, intent on destroying them, but they were rebuffed by the Negus, the Christian king of Axum. That group that fled to Axum only returned to Arabia in 6AH/627CE, by which time Muhammad and the rest of the growing Muslim community had settled in Yathrib/Medina.
The vast majority of Christians today adhere to a belief in the Trinity, but in the early centuries after Jesus' life, there were a variety of views about the relationship between the Father and the Son. One of the most prominent of these controversies was the belief promulgated by Arius of Alexandria, who stressed the uniqueness of God and rejected the claim that Jesus was co-equal with the Father. The popularity and variety of these views was so widespread that under the aegis of the Emperor Constantine, the Council of Nicaea convened in 325CE to established a single Christian creed, while simultaneously exiling dissidents who supported the Arian view.
The Nicene creed would eventually come to be accepted by the majority of Christian denominations (and it remains thus to this day), but it remained controversial and the matter was far from settled. For example, prior to Arius' death, Constantine pardoned him and invited him back into the fold of the church, while Athanasius (a principal figure in the First Council of Nicaea) was instead sent into exile. Several emperors in the fourth century also preferred Arian Christianity with its unitarian character to the Athanasian form and its belief in the trinity. The Nicene Creed eventually came to prevail however and a Biblical canon was established, declaring called other gospels apocrypha or heretical. The Arians had their own versions of the Council of Nicaea - the Council of Antioch convened in 341CE, and established its own Christian creed, led by figures like Acacius of Caesarea, Gregory of Alexandria, and most prominently Eusebius of Nicomedia (who baptised Constantine the Great on his death-bed).
All this is to say that while Trinity is overwhelmingly accepted by modern Christians, there were large periods of early Christianity where this was a matter of significant theological controversy.
The Qur'an confirms a belief in revelations given to other prophets: the Suhuf (a scripture revealed to Abraham), Taurat (the Torah given to Moses) and the Injil (sometimes translated as Gospel, bestowed upon Jesus). But The Qur'an also says these revelations were corrupted later by other people, who interpolated their own words and ideas into them. Accordingly, Muslims are expected to treat the Torah and the Bible with respect and reverence (some Islamic schools of jurisprudence even require that ablutions are performed before touching the text, just like the Qur'an), because in the Muslim view, fragments of authentic revelation from the divine may still be present in these scriptures, even if the totality of them are unreliable as a guide.
Lings, Martin (1983). Muhammad: His Life Based on the Earliest Sources. Islamic Texts Society.
Armstrong, Karen (2017). Muhammad: A Prophet for Our Time. HarperCollins.
Hanson, R. P. C (1988). The Search for the Christian Doctrine of God: The Arian Controversy. InterVarsity Press.