The line in context:
Among thise otherefolk was Criseyda,
In widewes habit blak; but nantheles,
Right as oure firste lettre is now an A,
In beaute first so stood she, makeles.
(Chaucer, Troilus and Criseyde I. 169 - 172)
Were there many big changes to the English alphabet in Chaucer's time? At what point did the English alphabet acquire the 26 letters and order we know today?
I'd just like to add as well that I'm just rehearsing for King Lear, which includes the line
Thou whoreson zed! Unnecessary letter!
was the adoption of "Z" a recent development too?
I suspect Chaucer is merely filling out his pentameter line with an extra iamb, and "is now" was the only thing that came to mind as sounding natural.
No, there were no changes to the alphabet in English relating to A at Chaucer's time. Some of his contemporaries were still using the thorn, but Chaucer used "th" for those sounds, probably because either his audience was closer to the Norman court and its Latin 26-letter alphabet or perhaps the Southeast Midlands dialect he spoke had already dropped its use.
The English had originally adapted the Latin alphabet with two addition characters, the thorn and the edh. The alphabetical order we have comes from late Roman antiquity, but even before the Roman state was founded, the Italic languages started with some sort of A-shaped letter, having adopted the Cumaean version of the Greek alphabet, which already had the A letter and sound value. The Greeks adapted their letters from the Phoenicians, where the analogous letter, aleph, had a similar-ish shape but a different consonantal sound value which was meaningless in the Greek language and replaced.