Wikipedia mentions that Misha Glenny notes their settlement in the Balkans inflamed tensions but I have the book and it doesn't go into much further detail.
Yes, the Ottoman Empire is a popular place for Muslim refugees from other places to go to ever since the 17th century, both voluntarily or by government design. Albanians and other Balkan Muslims migrated to Istanbul and the remaining Balkan lands held by the Ottoman Empire, Nogay Turks and Tatars fled Russia and settled in the Balkans after the Crimean War, Abkhazians and Circassians moved to the Balkans and Anatolia during the 1860s, and later more after the 1878 Ottoman war against Russia, and Caucasian Muslims like the Chechnyan and Dagestani migrated into Ottoman lands after Russian conquests in the Caucasus. Additionally, there were also Polish and Hungarian refugees who settled after the failed uprising in 1849.
As for conflicts caused by refugees, it certainly happened, for example in 1730 migrant Albanians fleeing the Balkans were said to incite unrest in Istanbul and the Porte prevents further migration to Istanbul by denying any Albanian whether Muslim or Non-Muslim a boat to Istanbul. However this were quite rare as the Ottoman government while encountered severe problems handling the refugees eventually managed to resettle them successfully with little conflict. The Circassian refugees you mentioned for example, were successfully settled in Rumelia, Bulgaria, and Thrace and were given home and agricultural lands, though i don't know if their presence further "inflamed tension". What did inflame tension though is the massacre and violent suppression of Bulgarian uprising by said Circassian irregulars who had settled there, but that happened around 14-15 years after their settlement. Hope that answers your question
Sources/Further Reading:
"Death and Exile: The Ethnic Cleansing of Ottoman Muslims 1821-1922" by Justin McCarthy
"Refugees, Exiles, and Other Forced Migrants in the Late Ottoman Empire" by Dawn Chatty
"Conversion and Apostasy in the Late Ottoman Empire" by Selim Deringil
"Osman's Dreams: The History of The Ottoman Empire" by Caroline Finkel
"The Ottoman Empire, 1700-1922" by Douglas Howard