I'm not sure about this medieval tradition you mention. I've never heard of anything like "pistols at dawn" sorts of duels in Rome between citizens settling personal or political disputes. There is a tradition of military duels that at least taps into the duels of the Homeric epics if they are not outright literary constructions entirely.

Livy has several stories of Romans fighting duels against enemies in various places. A contest was fought between two sets of three brothers from Rome and Alba Longa, the Horatii and the Curiatii, to decide which city would be hegemon without lots of killing. That seems directly related to the duel between Menelaus and Paris in the Iliad, in which they hope to resolve their conflict over Helen without killing many soldiers on each side.

Titus Manlius Torquatus was supposed to have fought a single combat against a Gaul in 361 before a general battle, for which he is supposed to have gained his cognomen. He is supposed to have later killed his son who participated in similar duels for breaking ranks against orders.

I am suspicious about these kinds of duels. I tend to think of these as etiological stories. However Goldsworthy has argued that the bulk of killing in Imperial Roman legions would have been done by a small percentage of soldiers probably fighting in a similar tradition - as action on the front line wanes space would open for these sorts of combats. We do hear from Caesar about Lucius Vorenus and Titus Pullo (not to be confused with the characters from HBO's Rome) who fought in front of the lines in Gaul, and at Pharsalus a guy names Crastinus who fought in front in a similar way. One of Caesar's soldiers in Africa was supposed to have swapped words with Labienus (formerly Caesar's legate but fought against him in the Civil War) before shooting his horse out from under him.

Rome's highest military honor was the Spolia Opimia, awarded only (apparently by the Augustan period) to a commander-in-chief who killed an opponent commander-in-chief in personal combat. Marcellus won that in 222 for killing his opponent Viridomarus in a cavalry charge. Other winners were Romulus (clearly mythological) and Aulus Cornelius Cossus (probably mythological, certainly this was a controversial tradition later). Crassus, the grandson of the famous Crassus, claimed the Spolia in 29 BCE, but was prevented by Augustus from claiming it. H. Flower I think still has the best article on the Spolia. She claims it was almost entirely an invented tradition of Marcellus.