So Washington DC was founded in 1790. The White House wasn't completed until 1972. But, the Treasury was founded in 1789. Where was the Department of Treasury before their Washington, DC location?
I think there may be some confusion here-DC was not the first capital of the US; New York and later Philadelphia were. So like other departments, it would have had offices in those cities. The State Department for instance occupied a series of rented quarters around New York and Philadelphia before the capital was transferred.