The O prefix is a corruption (linguists don't like the use of the word corruption because it suggests a degradation, I am choosing to use it because people understand what I am talking about more when i use it) of the word Ua from Irish Gaelic which means specially Grandson but might be more accurately translated to decedent. Thus the surname O'Neill means decedent of Neill (I'm not an expert of Gaelic, but I am going to hazard a guess that Neill is also a linguistic corruption of some original name).

So this all has to do with the evolution of surnames as a concept. When societies are small and isolated, people only need a single name to distinguish each other. As societies get bigger and and interact with each other people needed more ways to distinguish each other. All the strange ways this developed is fascinating, and I think what you are specifically asking about would require more knowledge about the clans of the British isles, which I don't know that much about

In general the reasons why we have names like this being so common is the transition from a Patronymic last name system to a Surname last name system in what is known as a Patronymic surname. The Patronymic system is a relatively simple system of distinguishing ones self further, you just take the name of one of your parents so rather then calling yourself "X" you call yourself "X" son of "Y" to help distinguish yourself from all the other "X's".

But as societies became even more complex and interconnected this just didn't cut it anymore, so systems of more unique last names were introduced and thus we get the Surname. However its much easier to keep the last name you already have, so thus have the O'Connors and O'Neills.

Keep in mind the Irish were not the only ones who did this as we also have names like Macdonald for the Scottish and Johnson for the English.