I'm specifically referring to foreign embassies based in Washington DC, USA, but if someone has a story about their country, that would be cool too. Were there embassies for Zaire, Czechoslovakia, the Soviet Union, East Germany etc.? Maybe even farther back, such as the Austro-Hungarian and Ottoman Empires? Were the buildings repurposed, demolished, or reassigned?
From a legal standpoint, in many of these cases, the principle of state succession is at work. I'll limit my answer to those cases where state succession is claimed.
The principle has a bit of an interesting history, and I did a little work on it during my grad school and law school studies, so there's probably a lot of historical debate we can have about why it developed as a theory, but that's separate from the legal aspect I feel much more competent to address. Basically, the question boils down to a question of sovereignty, the legal principle of ultimate jurisdiction over a given territory, and the people, places, and things within it.
At the root of it, the principle is that sovereignty does not lapse from government to government or regime to regime. Once established in a given place, sovereignty over the territory is always vested in somebody. As illustrative examples of the legal rationale, US Supreme Court Justice John Marshall laid some of these principles in Johnson v. M'Intosh and Cherokee Nation v. Georgia and described the seamless process by which sovereignty moved from the British Crown to the United States at independence.
So, since it is a principle at international law that sovereignty cannot lapse, there is a successor to each sovereign. The Russian Federation is the successor state to the USSR. A unified Germany is the successor to the the GDR. The DRC is the successor to Zaire. While there is often some wrangling about who counts as the successor, particularly in the case of the USSR-Russia transition, whomever the successor is generally takes all legal obligations and interests.
A historical example is the Russian Empire-USSR transition. The USSR fought the contention that the USSR was the successor state to the Russian Empire, and therefore was not obligated to pay the debts of the Empire, while the rest of the world argued the converse.
TL;DR: The successor state takes all the interests and obligations of the previously existing sovereign.
Fun fact: the iraqi embassy to the GDR (east germany) was a popular 'lost place' in Berlin. http://www.abandonedberlin.com/2010/06/party-at-saddams-house-abandoned-iraqi.html?m=1