Not exactly, they would often "abandon" their children. This was legally allowed by Rome that they could abandon a child up to the age of 7. Typically this would be in the manner of left to the wilds but also there were looser definitions where they would be kicked out of the home persey and typically picked up as a slave by farmers and other houses.

Later this rule would be struck down as the 'true blooded' Romans (people of the more traditional places such as Roma herself) were dying out and the Emperor at the time (I believe Constatine) decreed it was no longer allowed.

There was a previous post that contains some references on this that might help.

edit: typo