It almost always had to do with Gold Convertibility. Since the industrial era and the emergence of modern banking, if a Central Bank could defend it's currency's convertibility into gold at a set rate (or at least keep confidence in their ability to perform the conversion at a set rate) the value of that currency remained stable. This might sound like an arcane concept, but the mechanics were actually rather simple: in the past, you could show up to bank with paper banknotes and demand they be exchanged for the corresponding value of gold bullion.

Prior to the second world war, suspending or eliminating gold convertibility was a sign that an economy was in serious trouble. But starting from the 1950s, gold convertibility fell out of favor with all governments and their respective central banks, although some Central Banks to this day maintain a "Peg," that is to say actively defend their currency's convertibility with another currency. The United States ended gold convertibility fairly late, in 1976, but a number of countries still maintain a peg to the dollar.

Maintaining set exchange rates does have some benefits: it allows businesses to confidently plan ahead for the things they will have to buy from abroad (insulating end consumers from uncomfortable price shocks) and if the exchange is set at a favorable rate it can encourage foreigners to take advantage of a given country's cheaper goods and services. The most common case where a peg is sensible to implement is when a very small country with few resources is already highly dependent on a larger country's economy to either buy or sell most of their goods and services (e.g. because The Bahamas both buy very many consumer goods from the United States and also provide a service to very many people from the United States — tourism — such that a freely floating currency would be more of a nuisance than anything else, officials in the Bahamas have chosen to go ahead and peg their currency 1-to-1 against the US Dollar. Another example are currencies from oil producing economies that are often also pegged to the dollar, both for political and economic reasons).

But fixing a peg, be it to gold or another currency, eliminates important policy tools that modern Central Banks rely on to act as economic shock absorbers, like influencing interest rates. This is because all the money lent necessarily needs to be matched by reserves, which ultimately means there is less "slack" in the monetary and financial system. So while international currency markets are fairly stable under fixed exchange rates, the economy as a whole is subject to much more turmoil. Prior to the very famous financial crisis of 1929, less serious but nonetheless disruptive global financial crises had occurred every five years or so (sometimes more often!) in the decades prior: In 1907, 1901, 1896, 1893, 1890, 1886, and so on. In fact, the unique circumstances in the lead-up, outbreak, and aftermath of the First World War which caused the lack of any serious economic crisis between 1907 and 1929 was very anomalous for the Gold Standard era, and contributed to making the 1929 crash all the more disruptive. I wrote about the 1929 crash, and how the Gold Standard contributed to it, in this older answer which might interest you.

If you're interested in the modern history of global currencies, you can pick up B. Eichengreen's Globalizing capital: a history of the international monetary system (Princeton UP, 2008). It's the major source of the answer I offered here.