We tend to pronounce more familiar place names in a more anglicised manner - "Frahnce" and "PAR-iss" instead of "Frongss" and "Par-EE", foe instance.
Yet Calais, a city that was English for centuries, is pronounced "CAL-ay", pretty much as per French.
Just to be clear, while spelled identically, the premise of your question is incorrect: Calais is pronounced [kæleɪ] (kahl-ay) in (British) English, whereas in French, it is pronounced [kalɛ], which is pronounced similarly to ka-leh, the latter syllable rhyming with meh. In other words, quite differently from one another.
Nevertheless, it is true that the modern rendition in English is based on modern French rather than on a native or extensively anglicized form.
To answer your question, Calais was pronounced differently in English for most of history. The archaic pronunciation in English used to be [kælɪs] (kal-iss, rhymes with malice).
When concerning the Medieval period, one should always be careful to equate the language of the monarch or principal political entity of a certain realm with its entire territory. This is true for the Holy Roman Empire (which is readily and falsely equated with German), France and is also applicable to the lands controlled by the King of England; especially on the fringes.
With this in mind, it might be less of a surprise to find out that the population of Calais used to be primarily Dutch-speaking in the 14th century, even though it was politically English between 1347 and 1558. The Dutch language is the source of the archaic English pronunciation of Calais, as in Dutch the town was known as Kales, pronounced [kalɛs] (kah-less), which remains the common Dutch pronunciation even though the name of the town (which became monolingually French following the French Revolution) is now spelled Calais.
Language shifts tend to be very slow and complex processes, especially when the region is located far from the heartland of the realm it formally belongs to; or is physically separated from it. To give an example; the Channel island of Jersey has been in the hands of the English kings since 1066 and the last speakers of Jèrriais (a form of French) are only dying out today. Had the French government not implemented a ruthless policy of Gallicization throughout its territories following the French Revolution (which decimated regional languages such as Breton, Dutch and Basque) then Calais might have remained bilingual to this day. In that case, the chances are that English would have stuck to its 'traditional' pronunciation.