The Nobel Prize winning American economist Paul Samuelson famously wrote:

To prove that Wall Street is an early omen of movements still to come in GNP, commentators quote economic studies alleging that market downturns predicted four out of the last five recessions. That is an understatement. Wall Street indexes predicted nine out of the last five recessions! And its mistakes were beauties.

While that statement was made decades after the end of the Great Depression, and was made in jest, the heart of the sentiment is still accurate. Recessions are extremely hard to predict and, since the best way to make money from a drop in stocks is to short said stock, getting the prediction wrong can be financially ruinous.

Equally famous is the quotation by Irving Fisher made just one week before the initial crash of Oct 24, 1929 where he stated that New York stock prices had reached:

... what looks like a permanently high plateau.

Even days after the catastrophic crash, the Harvard Economic Society pronounced:

despite its severity, we believe that the slump in stock prices will prove an intermediate movement and not the precursor of a business depression such as would entail prolonged further liquidation

From those quotations, which are by no means unusual, we can see that recessions are hard to predict even when they are happening and it is worth understanding why this is so.

Let's imagine that someone could correctly predict recessions. That person would accurately be able to understand market movements in advance and have the ability to forecast the underlying asset trends that cause those movements. They would be able to accurately foretell when to short a stock to make immense profit as the market plunged.

You can see that in the case of the Great Depression where stocks continued to rise throughout the summer of 1929 even though the underlying fundamentals of declining factory production, rising unemployment and rising consumer debt were shortly to collide with drought conditions which would devastate the agricultural sector.

What finally broke the back of the stock market was a set of sell-offs on 24 & 29 October 1929 which quickly revealed that many of the shares which had previously high values were actually worthless. They were worthless for a variety of reasons but ultimately it was because no one wanted to buy them - the ever-rising bull market had run its course and the investors that were left at the end suffered the market correction.

In a rising market, it is difficult to hold short positions as the owners of the shares will require more money from the short investor each time the share price rises. In effect, there is no profit motive for predicting a recession unless you predict the timing exactly or have sufficient cash reserves to wait out a long period of behaviour which will go against your predictions. Or, as John Maynard Keenes wrote:

The market can stay irrational longer than you can remain solvent.

That is not to say that individual economists did not predict the Great Depression. Ludwig von Mises predicted in 1928 (in a time of unprecedented optimism regarding stock prices) that "every boom must have a bust.". He was not heeded for two reasons. The first is that the statement is glaringly obvious and the second is it was an incredibly vague prediction on the future behaviour of the market in the short-medium term. Anyone who bet on the crash in the middle of 1928 would have been well and truly bankrupt by late 1929 when the crash actually happened.

Finally, 90 years after the onset of the Great Depression and 11 years after the Great Recession of 2008, economists are still arguing over the exact causes of both and that is the real reason why no one can see a crash coming. If we can't predict it with hindsight, how are we going to predict it with foresight?

Or, as Nels Bohr put it:

I find it very difficult to make predictions, especially about the future.