The difference between Sunnis and Shi'a can be traced back to the first generation of Muslims. It boils down to one question: how was Muhammad, 'the Seal of the Prophets', to be succeeded as leader of the Muslim community? Similar to the question of Papal primacy in the Christian church, this was always both a political and a religious matter.

The Sunnis constitute the majority of Muslims (approximately 85%). Their answer to this question is: Muhammad cannot be succeeded as a prophet. The Muslim community can only select a khalifa or caliph (literally 'steward' or 'deputy'). The caliph is selected from Muhammad's tribe, the Quraish, by the process of consensus or ijma', and theoretically only has an administrative and law-enforcing function. The first caliph was Abu Bakr, Muhammad's father-in-law.

The example of first four of these caliphs, who all had a close relationship to Muhammad, and that of Muhammad himself, comprises the sunnah or custom for all later generations of Muslims to follow (hence 'Sunni'). Through the principle of ijma', which gradually came to mean consensus by legal scholars instead of the whole community, Shari'a law was developed through the sunnah.

Shi'a Muslims constitute about 15% of all Muslims. They are spread throughout the Muslim world, but are are particularly concentrated in Iran, Iraq, Syria, Azerbaijan and Yemen; see this map courtesy of the University of Texas.

The answer of Shi'a Muslims to the question of Muhammad's succession is that he appointed a spiritual successor, his son-in-law 'Ali. They are thus the Shi'a or 'partisans' of 'Ali, and reject the caliphate of Abu Bakr. 'Ali, unlike the Sunni caliph, did inherit Muhammad's spiritual abilities (his wilayah) becoming the first imam (leader). The wilayah was passed down through 'Ali's sons, all the way through to the disappearance of the twelfth imam, Muhammad al-Mahdi, in 974. This imam is believed to be a messianic figure who will return at the End of Time to reestablish justice on Earth. In the meantime, their will is interpreted by mujtahidun and ayatollahs.

The hostility between Sunni and Shi'a goes all the way back to the third caliph, 'Uthman. His relatively tolerant demands towards new converts to Islam led him to be assassinated by khariji, fanatic followers of Muhammad, in 656. Muhammad's son-in-law and the first imam, 'Ali, was appointed as fourth caliph. After a bloody civil war with 'Uthman's cousin Mu'awiya, 'Ali was also assasinated by khariji after agreeing to negotiations with Mu'awiya.

The latter ascended as the caliph of the new Umayyad dynasty. Since he wasn't of Muhammad's blood, this caused a bitter rift in Islam between the Sunnis, followers of Mu'awiya, and the Shi'a, followers of 'Ali's son Husayn. The Shi'a, who tend to be more spiritually focused (inner life), consider all Umayyad caliphs and most or all of their successors satanic usurpers. They tend to follow the early teachings more closely, and are more associated with martyrdom and revolution. The Sunni tend to be more focused on societal structure (outer life), though there is no hard and fast distinction. Their laws developed over time through ijma', particularly under the Abassid caliphate (8th-13th century).

So the difference and split is very old, and has grown and shrunk over time. Modern radicalization of Islam has more to do with its politicization, but that's a whole other story. But the initial difference is summed up by Bill Wurtz: 'the other kind of Islam: the one where we thought the first guy should have been the other guy.'

Sources:

Chris Horrie & Peter Chippendale, What is Islam? in Dutch translation (Utrecht: Kosmos, 1992).

Christopher Partridge (ed.), The New Lion Handbook: The World Religions (3rd paperback ed.; Oxford: Lion Hudson, 2007).

The Sunni/Shi'a divide begins at Muhammad's (pbuh) death. Arab tradition was to elect a leader democratically from among the prominent members of the community (broadly someone from the same tribe as Muhammad, the Quraysh). While funeral arrangements were being made, Abu Bakr (the Prophet's father-in-law and close companion) was elected as the first Caliph (deputy) of the community.

However, there was strong support for the idea of switching to a more direct hereditary system where 'Ali, Muhammad's cousin would inherit leadership, and so on through the direct male line starting with 'Ali and Fatimah. Shi'as firmly believe that Muhammad appointed 'Ali and his descendants by God's command as a matter of faith. Some believe that there are strong political underpinnings to this. The new muslim community had already overturned many Arab traditions, and many believed a hereditary system would bring longer lasting stability to a system plagued by tribal feuds. The word "Shi'a" means "partisan/follower" from "Shi'a 'Ali / شيعة علي " or "partisans/followers of 'Ali".

'Ali refused to recognize Abu Bakr's appointment as the first Caliph. Muhammad's daughter Fatimah ('Ali's wife) also refused to pledge allegiance. The Banu Hashim (Muhammad's clan within the Quraysh Tribe) also supported 'Ali-- As a side note, the Banu Hashim descendants are the Hashemites, the long time rulers of Mecca and the Hejaz; King Abdullah II of Jordan is the last Hashemite king and as such claims direct descendance from the Prophet Muhammad.

When the 3rd Caliph Uthman was assassinated, 'Ali was chosen as the 4th Caliph. The Caliphate was then plunged into its first full blown civil war. He was assassinated during prayer (a major sacrilege) and 'Ali's rival Mu'awiyah was elected Caliph. Husayn ibn 'Ali, 'Ali's youngest, would eventually bring up arms against the Caliph only for him and 72 of his family and followers to be killed in the Battle of Karbala in 680 AD. For many, this marks the definitive split of the two sides. Shi'as commemorate this battle yearly in a day of mourning that can often involve dramatic displays of sadness and self-flagellation.

At various points throughout history, Shi'as have faced persecution for the lack of support for the establishment. The tragedies that have occurred, such as 'Ali being murdered during prayer and Husayn and his family being killed, have become ritualized fixations and a way of linking themselves to the injustices experienced by the Ahl al-Bayt. Today Shi'as have developed an esoteric tradition using an Imam as the one true interpreter of the Truth and the Law. Some have even developed a messianic tradition (Twelver Shi'a Islam).

I would really recommend "The Arabs, A History" by Eugene Rogan for a detailed historical look.

I would recommend "Islam: A short History" by Karen Armstrong for a briefer historical summary with more emphasis on the religious aspect of the divide.

Really appreciate the responses guys- I see this is a heavy topic with lots of opinions, biases, and sides. Looking forward to learning more

The above answers all provide good information, to them I'd add the following points (all based upon the 2016 edition of Hugh Kennedy's The Prophet and the Age of the Caliphates, which is recommended both by the AskHistorians master list and the Oxford Bibliographies Online.

• One of the things that marks off Ali from the other caliphs was the strategy he used to achieve political legitimacy. In an attempt to cut through the division between the Islamic elite (the qurra) and the old tribal elite (the ashraf) by stressing the equality of all believers and the religious role of the caliph or imam. This concern for Islamic government and the equality of all believers was the hallmark of Proto-Shi'ism.

• Although modern Shi'ism requires that imams be descended from Ali, the movements out of which it arose were probably looser in their requirements, adopting a broader notion of the family of the prophet. The 'Abbasids, for instance, who were descended from Muhammad's uncle al-'Abbas, took over the caliphate and seemingly found widespread support. Later Shi'ites would condemn them as frauds.

• A key turning point came in the 786, al-Husayn ibn Ali led a revolt against the staunchy anti-'Alid Caliph al-Hadi. Ironically, the rebellion was such a failure that members of the family had todisperse (e.g. Idris ibn 'Abd Allah to Morocco, his brother Yahya to Daylam), which spread 'Alid loyalties throughout the empire.

• Sunnism and Shi'ism emerged as armed political groupings, as well as spatially segregated social groupings, in the tenth-century. It was in Baghdad that "Twelver" Shi'ism first emerged, with the key intellectual idea (developed most importantly by al-Shayk al-Mufid) being that of the hidden imam i.e. that the last publicly acknowledged imam (al-Hasan al-'Askari) had left a son who had remained hidden and would one day return to rule. In the meantime, he'd left some representatives to guide the Faithful in his absence. On a more practical level, Shi'ism became a distinct sect with three new elements: the public denigration of Abu Bakr and 'Umar; exclusively Shi'ite public festivals (e.g. celebrating Muhammad's acknowledgement of 'Ali as his successorduring the Farewell pilgrimage in 632); the performance of pilgrimages to tombs of the members of the 'Alid family. All three of these were public acts, and the first two necessarily excluded Sunnis. It is these features that Kennedy says distinguish true Shi'ism from the earlier supporters of 'Ali.