It's true. There's a good book in English about this by the Historian John Boswell called The Kindness of Strangers: The Abandonment of Children in Western Europe from Late Antiquity to the Renaissance. Boswell investigates the practice of child abandonment as well as the orphanages and other institutions that Christians developed to care for these children.

First, the phenomenon of abandonment was slightly more common in the Roman period than in later periods in Europe, but it occurred in Europe long after the Roman period. Boswell has argued that it may have occurred for between 20% and 40% of births in urban Rome. However, during the early modern era in Europe, it occurred at a rate of about 30% (Boswell 135). The practice is the subject of some confusion, many people conflating child abandonment with infanticide. Although sometimes abandonment was intended to result in the death of the child, usually in cases of birth defect or Greek plays in the theaters, for the most part abandonment was intended to give the child a better chance at life. Before and after Christians began making a deliberate show of rescuing babies, it was quite common for the abandoned babies to be rescued. In fact, usually they were left in conspicuous places in order to assure their rescue, for example near farms in hope that a male child might be taken in and made a farm-hand. It was even common for the children to be reclaimed by their parents if the parents eventually could care for the child (Boswell 74). Of course, some of the abandoned children did die. But the parents were not throwing them out in the hopes that they would; to the contrary, it was expected that kind souls would rescue the children.

There were some significant differences in the rescue of children by Christians. The pre-Christian practice of abandonment and fostering was more organic, whereas the Christians were actively searching out and preparing for these children. The Christians raised funds for these rescues, organized them, and set up orphanages and other institutions to care for lost children. Despite this, the children still often died. We don't know the rate of survival for abandoned children in Rome (Boswell 45), but we do know the death rates in later Christian foundling houses could reach up to 90% (Boswell 432).

The reason that Christians started doing this in a systematic way was because Charity was an important part of the Christian religion, and care for orphans and children was especially emphasized. As Christianity increased in prominence in Rome, its adherents began pursuing charitable acts both because they felt called to by their faith and because it made for good PR. I explained this phenomenon a bit more fully in this old comment (Full disclosure, a different thread I wrote on this post is where I pulled much of this answer from).