That's not a thing. You're thinking of metonymy which is when governments are referred to by their capitals, but nations aren't ever referred like that. Whenever you hear people talking about Moscow in that way they mean the Russian government, not Russia.

Also, Riyadh is the capital of Saudi Arabia, not Dubai. You're probably thinking of Abu Dhabi, which is the capital of the UAE.

I have no idea what you're talking about. Where did you hear people say that? Could you please provide an example of a sentence where the name of a nation is switched with it's capital?

This is a rhetorical technique, not limited to government; as such I doubt very much that you can reasonably assign a start date. It leverages the association of meaning between two objects to make one part stand for another, and is closely related to both metaphor and synecdoche.