While there's always more that can be said, I answered a similar question a while back about office workers in the 19th century that you may find interesting. Basically, yes, they likely went outside and used a public privy and that was the norm for men in D.C. until the 20th century. Edit: /u/cedric_hampton's comment below speaks to the presence of water closets in the capital building and is a great reminder that while I may know a lot of the history of women's bathrooms and bathrooms in schools, extrapolating too far isn't a good idea! That said, and I'm extrapolating based on writing about Harvard and Yale, it's also likely that the men simply went outside and relieved themselves wherever they wanted to. To borrow a bit from my older answer:

... there are letters and journals from tutors at Harvard and Yale in the early 1800s complaining about the students' poor behavior. They spoke about the messes boys made and the foul smell in their rooms. Likewise, the school inspectors I described earlier used similar language when describing schoolhouses; they were ill-kept, with a noxious smell. That smell was very likely urine from boys relieving themselves against the side of the school but norms were such the men didn't come right out and describe it in detail.

It's also reasonable for the spaces where men with means worked to have a designated area, usually separated from a larger room by a screen, with a chamber pot that could be used as needed. But such things were generally not described in detail by the men who used them. Meanwhile, it's worth noting that Jeannette Rankin, the first woman to serve in Congress, had to leave her office to use an outdoor privy a nearby public restroom (h/t again to /u/Cedric_Hampton) for the entirety of her term from 1917 to 1919. The first women's bathroom located near the floor of the House opened in 2011.