Conflict based on ethnicity in the former Yugoslavian region of the Balkans started mainly with the creation of nation states in the XIX century. The empires that ruled the area for centuries, Byzantine and Ottoman, were not ethnic entities. While there were tensions between Christians and non-Christians in the Ottoman empire there was no distinct ethnic conflict.

It was in the second half of XIX that more focus was given to creation of ethnic identities that will lead to bloody clashes in the XX century. The process of transformation from religious to ethnic identity was "less of a discovery than of creation of identity" (Eller). Like all European nation states had to legitimize their existence and their land claims and so did Serbia which has seen itself as Balkan "Piedmont" and wanted to create a Balkan state in similar fashion to the new Italian one. Promoting Serbian ethnicity to all Orthodox inhabitants of the Balkans was part of that process.

Similarly, Catholic church promoted united Croatian ethnicity from various catholic regions like Dalmatia, Slavonija.

As for the Muslims: "Interestingly, and significantly, although most Bosnian Muslims declined the invitation to identify and to affiliate as Croat or Serb, some accepted, and when they did, they more often chose Croat. Finally, and perhaps most interestingly and tellingly, since it also accords with early Bosnian behavior in regard to religion, Donia and Fine write that "Muslims changed from one national identity to another with about the same ease that an American might change polit ical parties" (1994, 111). Ethnicity in Bosnia, in this formative stage, was a remarkably fluid proposition, but this would end as social boundaries firmed and grievances acquired more substance and depth through time." Eller

So, the unfortunately ongoing ethnic conflict in the Balkans (well, at least the conflicted areas of ex-Yugoslavia) is continuation of the XIX century nation building and and it doesn't seem that it has roots in any earlier history. 

Sources:

Jack David Eller: From Culture to Ethnicity to Conflict

Mark Mazower: the Balkans

Marie-Janine Calic: History of Yugoslavia