Depending on what you mean by "cavemen", this question may be a better fit for /r/AskAnthropology. However, there have been lots of questions about shaving in this sub, so you may be interested in the FAQ:

• On the history of facial hair and the shaving thereof