In the early 19th century, it seems like black is almost exclusively worn by Protestant priests, whereas the worldly formal attire seemed rather liberal and colorful, but at the eve of the century, it seems like it's one of the most, if not the most popular color for formal attire. How and why did this happen?
This one is not too hard actually. It was largely due to developments in the chemicals industries. These developments provided two things, firstly, the dyes themselves became brighter due to being produced in an artificial and more concentrated form, allowing a much greater range of colours not evident in the Georgian period. The second was the discovery of multiple mordant (dye fixing chemicals). These mordants meant that the colours used to dye fabrics would better survive multiple washings, prior to this, buying a black garment would mean that after 10 -15 washes it would be more of a dull grey, at this point the garment would have to be un-stitched at the seams, and re-dyed before being worn again.
The advent of these techniques meant that you could buy a black garment that stayed black over time, and at a price that was much more accessible to all.