It is the English translation of the French phrase ""Qu'ils mangent de la brioche." It is from Jean-Jacques Rousseau's book "Confessions." The full quote from the book is:

Enfin je me rappelai le pis-aller d’une grande princesse à qui l’on disait que les paysans n’avaient pas de pain, et qui répondit : Qu’ils mangent de la brioche

Translated as:

Finally I recalled the stopgap solution of a great princess who was told that the peasants had no bread, and who responded: "Let them eat brioche."

Brioche being a bread enriched with butter and eggs (cake like) and was more expensive then regular bread. Thus the quote is demonstrating either a complete lack of compassion for the poor, or complete lack of understanding of poverty or a mixture of both.

The quote is commonly associated to Marie Antoinette, however seeing as how Marie Antoinette was 14 and not in France when "Confessions" finished being written (published in 1782 but was completed in 1769)