Food prices fell. The US, Canada, et al were highly productive. The estates basically made their money off of food production. When the price of food fell, they were screwed.

Also the rise of industrialization and the modern economy created a bunch of new rich people unconnected to the old aristocracy. More importantly, they upped the stakes on what it meant to be 'rich'. Food production could only manage to make you rich by 18th century standards. The 19th century had higher ones.

So food prices are falling, lowering incomes. And peasants are leaving the farms to go work in factories for better and more regular paychecks, lowering farm output and the income of the estate even farther.