We can infer from parts of the New Testament that there was a debate among Jews in the first and second centuries about whether Jesus was the messiah or not

Critically, while you can infer that amongst a select number of Jews there may have been such a debate, keep in mind that the NT is specifically designed to sell Christianity and is interested in playing up Jesus/Christianity's impact on society. This isn't necessarily the historical reality.

Historically, the broader Jewish world seems to have paid no particular significant notice to Christians and Christianity for quite some time. Christianity had an interest in competing with its parent religion to achieve legitimacy, and needs to prove itself to everyone. But for Judaism, it simply wasn't relevant or impactful at the time (they're far more interested in worrying about Grecco-Roman Paganism), and we only see a few eventual moves to keep Christianity out of the Synagogues (which were generally categorized under general prohibitions against Heretics).

While it may not be apparent now, at the time, it was just one of dozens of fledgling sects, breakaways and messianic claimants, and Christianity quickly became a religion popular with and dominated by non-Jews, whom Judaism was not particularly interested in selling its religion to.

What do non-biblical Jewish sources say about how Jews who didn't think Jesus was the messiah responded to Jesus and early Christianity?

As mentioned, early Jewish literature by and large doesn't mention Jesus or Christianity. The exceptions which are typically referenced include Josephus, who has a few throwaway references -- one in particular is extremely suspicious for its veneration of Jesus and is thus of questionable authenticity -- and the blessing against the Heretics, of which early versions were evidently written with Christians explicitly named.

When you get a bit later in Rabbinic texts, e.g. the Talmuds, you see a number of evident references to Christians where the attitudes range from strong eye rolls and satire, to condescension. It seems evident to me that they did not seriously consider the Christian groups to be worth worrying about at that time (certainly not in the literature from the Sassanian empires where Christianity had no particular role in society and Zoroastrianism and Manichaeism was much more dangerous and culturally seductive).

How did Jews in later centuries (fourth, fifth) confront the growth of Christianity?

Jews, not being in a place of power in the Roman Empire, had no particular ability to confront Christianity. As the Roman Government became Christian, what you do see is a push for punitive laws against Jews (and eventually Pagans as well) to force a transition to Christianity.

Here's a nice study that covers these issues in depth for the later: The Persecution of the Jews in the Roman Empire (300-428), good quote from the conclusion: "...Jews in the Roman Empire experienced a series of misfortunes during the fourth century which reduced them from privileged citizenship to oppressed exile. These misfortunes occurred mainly as the result of Christian animosity, propaganda, and persecution."