I can see how the spellings of Peking and Beijing might both be derived from hearing the same word slightly differently. Xi'an and Chang'an, however, are drastically different, and Chiang Kai-shek and Jiang Jieshi are almost entirely different. In the latter instance, how could one romanizer possibly hear two hard k's where the other doesn't? The only consistent thing between different systems seems to be the number of syllables. Why would separate romanization systems using the same alphabet to represent the same sounds have so much variance in which letters they choose to represent those sounds?
So in general terms, one of the key reasons for variation in Romanisation is the difference between two major standards: Wade-Giles, which was developed by Anglophone linguists at the end of the 19th century and remained standard until the 1980s, and Hanyu Pinyin, which was developed by Chinese linguists in the 1950s and which replaced Wade-Giles as the international standard, except in some writings by academics trained in Wade-Giles, and also on Taiwan. The main difference is in which characters are chosen to represent certain sounds. Wade-Giles is an attempt to apply English phonology to Chinese – and specifically the Beijing dialect – whereas Pinyin simply maps certain sounds directly onto certain letter combinations. So for a table of examples:
| Character | IPA (Beijing) | Wade-Giles | Pinyin |
|---|---|---|---|
| 清 | [tɕʰiŋ] | ch'ing | qing |
| 人 | [ʐən] | jen | ren |
| 工 | [kʊŋ] | kung | gong |
| 別 | [pje] | pieh | bie |
So to reiterate, Wade-Giles approximates the actual pronunciation using English phonology, whereas Pinyin has its own set of rules.
However.
None of the examples you've listed are the result of Wade-Giles/Pinyin differences.
Beijing/Peking:
'Peking' is the form that was popularised by Jesuit missionaries in southern China, and likely a) derived from the Nanjing dialect of Mandarin, and b) employing Romance phonology. Because of b), the 'e' in 'Pe' (for 北) was intended to approximate /e/, rather than the /i/ that is often (erroneously) used by English-speakers; while because of a), the rendering of 京 begins with /k/ rather than /tɕ/. Simple familiarity kept the transliteration in use. By way of analogy, 'Vienna' neither looks nor sounds like 'Wien', but that's because it was transmitted into English via Italian and French, rather than directly from German. Similarly, 'Peking' comes to us from Nanjing dialect via French and Italian, rather than being a direct transcription of Beijing dialect. And just so you know what that might look like, in Wade-Giles the Romanisation would be 'Pei-ching'.
Xi'an/Chang'an
Both of these are in Pinyin, and the cause for this is very simple. The city has changed names. The city of 長安 Chang'an was the capital of several dynasties but was sacked in 880 and abandoned as capital two decades later. It became known as 西安 Xi'an during the Ming Dynasty, when it was refortified and rebuilt. Beijing has similarly gone through some changes of name: on the accession of the Ming in 1368 it was known as 北平 (Beiping/Peip'ing), but renamed to Beijing when the Ming moved their capital there in 1403; subsequently, between 1928 and 1945, when the Republic of China moved the capital south to Nanjing (and latterly to Chongqing), it reverted to Beiping.
Chiang Kai-Shek/Jiang Jieshi
In this case, we're not even talking different dialects and Romanisations of Mandarin, but different languages altogether. While 'Chiang' is the Wade-Giles for his surname, and the direct equivalent of 'Jiang', 'Kai-Shek' derives from the Cantonese pronunciation of 介石 (IPA: [kāːi.sɛ̀ːk̚]) rather than the Mandarin (IPA: tɕjê.ʂɻ̩̌).