What was so different about Manzikert was that it lead to around 10 years of civil war, first between Romanos IV and Michael VII/Andronikos Dukous, and then a host of other contestants.

It is important to note that Manzikert did not result in the destruction of either the Byzantine army there, or the military establishment. In fact, Andronikos Dukous (who betrayed Romanos by withdrawing) kept the rear-guard intact and fell back so it was hardly as catastrophic as popular history presents.

However, civil wars usually involve troops being redeployed, either to confront pretenders or because they were recruited by pretenders. This not only depleted the supply of trained soldiers, but also led the provinces exposed to attack. Civil wars also cause a breakdown in administration, so the government no longer has regional authority.

This small window of instability was taken advantage of by a variety of Turkish nomadic peoples. Not only did they raid, they moved into the now vulnerable provinces and, being the only military force present, could establish themselves as a new elite. They were also recruited as mercenaries by various Byzantine factions, resulting in yet more Turkish nomadic groups entering the empire.

This was one of those funny accidents of history. A perfect storm, so to be speak. Had the Byzantines not been busy fighting civil wars, or had the Middle-East not been experiencing the collapse and establishment of new regimes, or had there not been several major population movements, the Byzantines would never really had lost Anatolia.

Sources

The Alexiad, by Anna Komnena

Chronographia, by Michael Psellos

The Byzantine Empire, 1025-1204: a Political History, By Michael Angold

A History of the Byzantine State and Society, by Warren Treadgold

As the other writer here pointed out, the Byzantines were not reduced in actual numerical strength much by this defeat. The loss at Manzikert, arguably, killed fewer Byzantine soldiers than in the times when Amorium was sacked by the Arabs in 838.

The problem was that two centuries before, the Byzantines used a well organized national army loyal to the Emperor and the Empire. And that army was recruited among the Asia Minor Christian populations, who were fighting for their own land.

The contrast at Manzikert was considerable, with the Byzantine army mostly staffed by mercenaries, including pagan Turks from the European side of the Black Sea, and few men who were local recruits. The mercenaries were loyal to their pocketbook. Byzantine taxes had gone up, with the money going to the capital, while Byzantine fortresses in Asia Minor were often unmanned. Turkish scouts of Alp Arslan even found fortresses "manned" by manikins.

The biggest factor was that after the defeat, the peace treaty was rendered null and void by the deaths of Romanus IV Diogenes and Alp Arslan. The civil wars of the Byzantines allowed one Turkic horde after another into Asia Minor; the Turks did not tax the locals more than Constantinople had; and the Muslim Turks soon possessed the interior of Asia Minor, which they held on to permanently.

Socially it seems that the Byzantine culture in inner Asia Minor was less Greco/Roman than local. Writer Paul Wittek said: "It was really only the Byzantine varnish that vanished...to be replaced later on by an Islamic one. The local substratum survived." And the locals did not see the Turks wholly as foreigners, having met Muslim Turks in the East and pagan ones from Europe, operating as mercenaries in Byzantine armies.

So religious and linguistic transformation started.