In the wake of Bull Run, 1861, there certainly was general concern that Confederate troops might take Washington, though we now know from hindsight that First Manassas was a near-run thing, and the shaken Confederates were not in the mood or condition for a massed assault against Washington. Afterwards, the much-famed Army of the Potomac was originally formed as the Department of the Potomac, commanded by McClellan, which was entrusted with the duty of protecting D.C. while building extensive fortifications around the city. Afterwards, a standing garrison was kept in the city (usually by rotating front-line units for garrison duty), and the defensive lines ordered by McClellan was continuously bolstered, numbering some 68 major enclosed forts, 93 detached batteries, 20 miles of rifle pits, 1501 gun emplacements, 807 guns and 98 mortars and so on.

So, barring a 'what if' scenario right after First Bull Run, Confederate forces certainly could have passed by close to Washington in fast-moving, small sized columns (as J.E.B Stuart famously did in the Gettysburg campaign). However, they had to amass a sizeable, well supplied force with a siege train if they wanted to genuinely 'take' the capital. Which they couldn't afford. Not to mention the AoP out in the field - the army was created to protect the capital, after all. If you see the course of the Gettysburg campaign, you can see that the AoP is constantly positioned between the AoNV and Washington. Even if they wanted Washington (Lee's papers suggest otherwise), the logical conclusion was to destroy the AoP in the field beforehands (Which was Lee's main objective anyways). Now, IF the Confederates had won Gettysburg (and secured enough supplies from the North), Lee MIGHT have considered (as Longstreet suggested) a thrust against Washington, but as they didn't...

Afterwards, in 1864, when garrison troops were stripped from the Department of Washington for Grant's Overland Campaign, the defensive garrison of the Washington forts (XXII corps) were considerably weakened, not to mention the AoP being absent, as they first participated in the Overland Campaign, and then were busy around Petersburg. The Confederate Second Corps, AoNV under Jubal Early grabbed this opportunity and maneuvered to attack the capital defenses, but this was more of a ploy to direct Union troops elsewhere from Richmond. While the XXII corps garrisoning Washington were certainly comprised of undermanned batteries and second-line troops, it is highly doubtful that Early's troops could have pressed on to actually take the capital, not just threaten it. To quote Early himself (emphasis mine);

[ Finding this, I asked him about the character and strength of the works around Washington, and he said that they were not very strong, as they were nothing but 'earth-works'. I then asked him about the strength of the troops inside of those works, and he stated that there was not a large force in them—not more, he thought, than 20,000 men. Knowing that earthworks in the then state of the science of war were regarded as the strongest that could be made, and that such works, defended by 20,000 men, would be impregnable as against my force, and not feeling very much encouraged by the information given me, I nevertheless replied to my informant that if that was all they had to oppose to us we would not mind that. I have no doubt that some of my men, even after they were made prisoners, did what is called some 'very tall talking' about my strength and purposes, and doubtless such boasting on their part contributed in no small degree to the state of bewilderment of my opponent in the subsequent campaign as to my strength and the success of my efforts to battle him for so long a period. Washington was indebted for its safety not alone to the strength of its defenses and the troops that were in them before my arrival, but two divisions of the Sixth Corps from Grant’s army and a portion of the Nineteenth Corps arrived before or simultaneously with my arrival in front of the works. When I speak here of my arrival I mean, of course, the arrival of the main body of my force.

My troops did not all get up and Into line before 4 o’clock, and my leading brigade was not in line before 2 o’clock: so that, In addition to the troops already in Washington before my arrival, I would have had to encounter the two divisions of the Sixth Corps and the part of the Nineteenth Corps that had arrived if I had attempted to enter Washington. The proposition, therefore, that I could have successfully made the attempt at any time after my arrival is simply preposterous. If I had been able to reach Washington sooner, Grant would have sent troops to its rescue sooner, and hence there was never any prospect of my capturing that city. It was not Gen. Lee’s orders or expectation that I should take Washington. His order was that I should threaten that city, and when I suggested to him the probability of my being able to capture it he said that would be Impossible. It was my own conception, that of undertaking the capture, but the feasibility of that depended upon my finding the city very insufficiently defended. On the night of the 11th, being unwilling to surrender the idea of capturing the Federal capital. I gave an order for the assault at dawn on the 12th, but a dispatch received during the night, stating the arrival of two corps from Grant’s army, caused me to examine the works at the earliest dawn of the 12th, when I found them so strongly manned as to preclude all hope of carrying them, and I therefore countermanded the order for assault, I remained In front of the works, however, during the 12th, with the purpose of retiring at night, and gave orders accordingly. All my movements during the day were mere demonstrations to amuse the enemy until the time for withdrawal arrived. ]

In any case, VI corps redeployed quickly to stop Early, just as he stated. Early's march towards Washington was checked at Monocacy Junction, and stopped at Fort Stevens, on the northeastern perimeter of the city defenses. But the ploy certainly had the desired effect, as VI corps and XIX corps were dispatched to bolster the defense of Washington, and Sheridan had to lead the Army of the Shenandoah in pursuit of Early until Cedar Creek.

So, to sum it up, Washington D.C. was (generally speaking) quite capably defended during the entire war, with the AoP in the field and static defensive works thrown up around the entire perimeter. Lee knew that in order to take Washington (which never was a major war goal, by the way - the Confederate main objective was secession, remember), he had to first destroy the AoP (which was a major war goal). Confederate generals knew better than to just throw the AoNV against well-prepared fortifications with the AoP in their back. Afterwards, when the AoP wasn't around the vicinity, (it was participating in the Siege of Petersburg, which directly threatened the Confederate capital of Richmond) Jubal Early's raid was designed to threaten D.C., thereby drawing Union troops away from Richmond, but he didn't recieve orders for a siege, either.

Some sources:

National Park Service, Civil War Defenses of Washington (https://www.nps.gov/cwdw/learn/historyculture/index.htm) (accessed 2022)

Edwin B. Coddington, The Gettysburg Campaign: A Study in Command (1997)

Gary W. Gallagher, The Shenandoah Valley Campaign of 1864 (Military Campaigns of the Civil War) (2006)