Especially since until the time of the Shogunate, Kyoto had been where the Emperor lived for hundreds of years. The Edo plain was just marshland. Yet over 400 years, the Kanto dialect has become the formal/standard Japanese dialect, associated with government officials, the Imperial Family and the NHK; while Kansai-ben has been associated loud and boisterous characters, or owarai (comedy duos). How did such a dramatic change happen?
For instance, the phrase "are you serious...?/are you kidding me...?" can be translated into standard Japanese (i.e. the Kanto dialect) as the formal and fussy 真剣ですか. But in Kansai-ben, it's ほんまかいな.
The distinction here is between Kansai and Kinai. Kansai, you are correct, is commonly associated with Osaka-ben and earthy boisterous Osaka culture. That is mostly a legacy from the Tokugawa period when Osaka became a huge commercial center dominated by merchants.
Kinai is kind of a subregion of Kansai but without the down-to-earth overtones; it's essentially Kyoto. Kyoto-ben is supposedly more elegant and refined than Kansai-ben because it more closely resembles the Japanese used by the imperial court.
Kanto-ben became standard Japanese because 1) Edo was the political center of Japan during the Tokugawa period and 2) the Meiji government established a new modern capital in Edo/Tokyo and declared Kanto-ben "official" Japanese.
A few points:
Well, during the Tokugawa Shogunate, Edo was the center of the government, population, and a lot of business. It's where people looked to for sophistication and power, so that's the speech people interested in those issues emulated. The imperial family was deliberately kept isolated and without power in Kansai, and virtually nobody knew how they spoke even to imitate them. The way the imperial family spoke at that time was so different from the way anyone else spoke, that it was almost incomprehensible. It was of a completely different category of speech so it was not influential on anyone's daily speech. I'm not sure why you say that the Kanto dialect is associated with the imperial family. They are associated with their own formal speech.