Depends how loosely you want to define "colonize," "annex," and "empire."
Here are the ones that come to mind:
Russia: Lost (with German help) Poland, Finland, the Baltic States and Ukraine during 1917-20, re-annexed the Baltic States, and part of Poland and Finland(as the USSR) over 1938-45.
(While we're at it, there's also the Crimea, but that passes the 20 year rule)
Germany: Lost a bunch of territory to Poland in 1919, took it back in 1939.
Britain: has so far invaded Iraq three times. The first invasion(in 1920) went about as well as the others.
China: Lost control of Tibet, Xinjiang and Inner Mongolia in 1911; reacquired them between 1947-59.
You'll likely find the episode of the AskHistorians podcast concerning the Pueblo Revolt interesting.