They didn't necessarily have to do anything: umbilical cords naturally drop off 2–10 days after birth. As far as I'm aware there's no pressing medical reason to cut them, it just means you don't have to carry around a dead placenta for a week. Some people opt to leave the cord attached with no apparent ill effects.

If they did want to cut it, Stone Age didn't have to start gnawing at it. Our ancestors have been perfectly capable of producing a cutting edge using chipped stone for over 3 million years. Sharp edges, too: chipped obsidian has an even finer edge than steel. Surgeons use it to this day.

I'm also pretty sure that they were capable of tying knots without making them "giant", but in any case that's a myth: umbilical cords seal up before shortly after birth, which is what forms your belly button. Doctors just trim off the rest of the cord, they don't tie it.

Of course, there's no hard evidence for exactly what prehistoric people did with umbilical cords. That kind of behaviour leaves no trace in the archaeological record.

I don't believe that "caveman" has useful meaning in history or anthropology. I think you're intending something like "stone age peoples".