I often hear anabaptists argue that infant baptism was not a practice by the much-romanticized "early church" (which, as I understand it, means the 1st century Christian church). How legitimate are those claims? Was baptism understood as an adults-only ritual in the time of Jesus and the apostles? Are anabaptist arguments about "believer's baptism" as a historical practice of the early church defensible from a secular historian's perspective?
This is a highly controversial topic. There is no clear example of infant baptism in the New Testament. There are some examples of households being baptized, but it is not known whether those households contained children (or if the children in them were included in baptism). The oldest baptism manual--the Didache (dates to the first century)--contains no provision for infant baptism. It seems to assume that those baptized would be adults and that they would be baptized by immersion. Of course, this is an argument from silence. There is no argument against infant baptism in the first century (which isn't surprising if the practice was unknown or rare). Moreover, several of the earliest references to infant baptism are ambiguous or negative. For instance, Tertullian's work on baptism (c.200) advised that baptism for children be delayed.
There are other examples of church fathers practicing and advocating infant baptism. Certainly, by the third century the practice was very widespread, but not quite universal. So, for instance, the father of Basil the Great and Gregory of Nyssa (they were brothers), was a bishop, but they were baptized as adults.
So, yes, credobaptists (a better word than anabaptists) have a strong argument. This historian's best guess is that infant baptism began to spread rapidly throughout the Christian world during the first few decades of the second century. It likely had its origins in the practice of emergency baptism for sick newborns though likely to die. But many will disagree with this reconstruction of events.