It's a byproduct of British colonialism. During their process of colonisation, the English settlers often took Irish names - of people and places - and Anglicised them. For example, the capital of the Republic is Dublin, based on the viking settlement that used to be there called Dubhlinn (Blackpool if translated literally). More than that, though, while Irish was never prohibited in general (contrary to many popular myths), English was the language which dominated the Irish education system and civil service, and English was taught exclusively in the education system until 1871. There was major social pressure from the Catholic Church as well to discontinue the usage of Irish and they advocated against people speaking it until around the 1890's.

That attitude continued among a huge section of Ireland in spite of the Cultural Revival at the turn of the 20th century because employment opportunities were to be found in the Anglosphere - the United Kingdom and the United States, so Irish people were encouraged to learn English for when they would "inevitably" emigrate.

Now to the thrust of your question; Irish people's names are spelled with such variety because they weren't originally in English. They had to be Anglicised at some point and the method in which that was done wasn't done so consistently. O'Neill vs O'Neal vs O'Neil for example, would translate roughly back to Ó'Néill (grandson/descendant of Néill). Another example could be Piers v Pierce v Pearse v Pearson, which would go back to Mac Phearais (or Nic Phearais if they were a woman).

That's not to say Irish names in the original format are extremely consistent either. Ó'Néill could be Ua Néill, or Uí Néill, or Ní Néill but that's defined by rules - depending on stuff such as a person's gender.