Yes and No. It worked almost like duels in the western sense, only to preserve the dignity of samurai class, but the rules on "Kirisute Gomen" was very strict according to Kujigata Osadame Gaki, the 81 laws of justice, written in 1742.

If a samurai had ever decided to execute a civilian on site,

1. One has to notify at the office immediately
2. Whatever the reasons were, one has to face a mandatory 20 days house arrest.
3. One's sword will be confiscated until court decisions as an evidence (which was very dishonorable to any samurai not to have swords on their hips).
4. One has to have a witness to prove that the striking was a self-defense act against civilian braking the code, or rude behaviors.

If there was no witness or you couldn't prove yourself, you must face seppuku, and ended up dishonoring your family name, so Kirisute Gomen had rarely happened. The infamous Namamugi Incident involving British in 1862 is well-documented so maybe you can read about it.

:: Side question, were swords actually tested on convicts?

As a part of execution, convicts on death rows had occasionally faced the cut. You can read that on wiki by typing "Tameshigiri".