How did having white clothes mean you were poor because you couldn't dye the clothes turn into having white clothes being a symbol of wealth, like Italians with white shoes and a big white Cadillac.
Another question stemming from this : Did Roman senators actually wear togas? What color were they? Follow up question if they were white : Why were they white, instead of purple, or any other color that would require a dye?
Poor man's clothes weren't actually white, they were just the natural off-white/taupe/mottled unbleached color of the natural fiber. If you want actual white clothes you'll have to soak fabric in a caustic ammonia solution (made from stale urine) and spread it out in the sun for a really long time. There were some cheap dyes available but only in dull colors. Eventually chemical bleach and synthetic dye became readily available, but laundry still had to be done the old-fashioned way. Naturally poorer people chose more easy to care for colors instead of white.
I'm not sure if the modern gaudy Italian fashion is in any way related to this though.
I would guess it had something to do with ability to wash the fabric. Once everybody could afford color, being able to keep your clothes absolutely spotless proved that you weren't forced to labor in them all day. Like having manicured nails.