The Armenians were always pretty distinct from other kinds of Christians. They weren’t really part of the Roman/Byzantine Empire. Sometimes they were within the borders of the Empire, but they were never fully conquered or integrated. Theologically, there was a schism between “Chalcedonian” and “non-Chalcedonian” churches, based on which ones accepted the results of the Council of Chalcedon in 451. The council discussed the nature of Christ’s humanity and divinity and concluded that Christ had one nature that was both human and divine at the same time. The churches that disagreed with this argued that Christ had only one divine nature, which happened to manifest itself in human form. Seems like splitting hairs, maybe, but it was a major theological divide at the time, and the Armenian church was among the dissenters. After 451 (or really 506, when the Armenians officially adopted the non-Chalcedonian position), they developed independently of the Chalcedonian church in Constantinople.

The Armenians may have arrived in Jerusalem as early as the 4th century, when the Armenian kingdom converted to Christianity. They were sometimes persecuted by the Roman/Byzantine Empire. They didn’t have their own bishop in Jerusalem until the Muslims conquered the city. Caliph Umar recognized them as a separate community, and an Armenian patriarch was appointed for the first time.

The community was still there when the crusaders arrived, and Armenian sources depict them as very enthusiastic about crusader rule - supposedly the patriarch even joined in the massacre of the Muslim inhabitants in 1099. The crusaders already had a good relationship with the Armenians in northern Syria and Anatolia, and two of the first three kings of Jerusalem were married to Armenian princesses. Through Morphia of Melitene, wife of King Baldwin II, the Armenians had a lot of influence in government, art, and architecture. Even though the Armenians were theologically different from the Latin Catholic Church of the crusaders, they got along quite well. This was probably the period where a distinct Armenian quarter took shape, built around the monastery/cathedral of St. James in the southwest part of Jerusalem, near Mount Zion. The current cathedral of St. James was built during the crusader period in the 12th century.

Under the Egyptian Mamluks in the 14th century, the Armenians were allowed to build a wall around their quarter, and they were exempt from taxes that the other inhabitants of the city had to pay. The four modern quarters really originate in the Ottoman period in the 16th century. The Ottomans allowed the Armenians to keep their already-existing walled quarter rather than absorbing them into the rest of the Christian quarter. The population varied wildly though and at some points there were only about 100 Armenians there, but sometimes there may have been many thousands.

Basically, there’s an Armenian quarter simply because the Armenians are one of the most ancient Christian communities, they moved to Jerusalem very early, and they distinguished themselves from other Christians in language and doctrine very early on as well. They had simply always been in Jerusalem as far as Jerusalem’s later rulers were concerned, so why not let them be?

(I understand that in modern Jerusalem there are several difficulties faced by the Armenian quarter but I will leave it up to a modern expert to comment on that!)

Sources:

Adrian J. Boas, Jerusalem in the Time of the Crusades: Society, Landscape and Art in the Holy City under Frankish Rule (Routledge, 2001)

Victor Azarya, The Armenian Quarter of Jerusalem: Urban Life Behind Monastery Walls (University of California Press, 1984)

Avedis K. Sanjian, The Armenian Communities in Syria under Ottoman Dominion (Harvard University Press, 1965)

Amnon Linder, “Christian Communities in Jerusalem,” in The History of Jerusalem: the Early Muslim Period 638–1099, ed. Joshua Prawer and Haggai Ben-Shammai (New York University Press, 1996)