Claudius was born to immensely wealthy parents and had a place in the line of succession in a dynasty that at that time was still in its foundational years. It is also probably worth saying his physical disabilities are not the sort that would be immediately apparent at birth. All in all, he was not a prime candidate for infanticide.