There have been various reports of unusual weapons being used in duels. Generally speaking, the challenged party had a choice of weapons and conditions. I can give a few documented examples of 'non-standard' duelling conditions:
Following his infamous 1856 beating of Charles Sumner after Sumner had denounced slavery in Kansas, Rep. Preston Brooks was goaded by Rep. Anson Burlingame of Massachusetts into challenging Burlingame to a duel. Burlingame, a famous marksman, unexpectedly and eagerly accepted, choosing rifles and the Naval Yards on the Canadian side of Niagara Falls (presumably the two men would have shot at each other from either side of a drydock or some such). Brooks avoided death, but not humiliation, by claiming that he could not cross 'hostile territory' (ie, the North) to reach Canada. Evidently ships did not exist in antebellum South Carolina.
The 23rd July edition of the Northampton Mercury reported a 'novel duel' in Paris, in 1808, Monsiers Granpree and Le Pique fought a duel over the virtue of Mme Tirevit, a beautiful opera singer. The duel was fought in hot-air balloons, with each duellist attempting to penetrate the other's balloon with a blunderbuss at a height of 900 yards. M. Granpree won the day, bursting Le Pique's balloon and 'causing it's rapid descent', at the end of which Le Pique and his second were 'dashed to pieces' upon the roof of a house. Granpree 'mounted aloft in the grandest style' descended safely.
There is an anecdotal story that Rudolf Virchow, the famed German medical scientist, when challenged to a duel by Otto von Bismarck, chose sausages as the weapons of choice. Two sausages would be procured, one of which had been infected with the parasite Trichinella, which Bismarck would eat uncooked, and one cooked, which Virchow would eat. Bismarck declined, because contracting trichinosis is the wurst. Virchow was a great advocate of public health, and he frequently challenged people who dismissed this as unimportant to eat contaminated food.
Whilst sword and pistol were the 'traditional' weapons of a duel - pistols were also the safest for novices - there does seem to have been a healthy number of odd choices of weapon made, from Burlingame's rifles to Virchow's sausages.
Wow. This is unusual. So far we've removed 3 answers that are nothing more than "I heard a story about X but have no idea if it's true".
A moderator reminder to anyone considering submitting an answer in this subreddit: you should have enough expertise in a subject to actually have some degree of certainty whether claims have any basis in reality, and further, to supply the OP with detailed and supported accounts, rather than half-remembered or dodgily-sourced rumours. Please be mindful of which subreddit we're in here, and strive to comply with the rules for good answers. Thanks all!
It depends on the time period, and what you consider unusual. During the Italian Renaissance, for example, sword duels might be fought with the accompaniment of a metal gauntlet on the other hand, which allowed the wearer to grasp the opponent’s sword, or with the use of a cloak which could hide the movements of one’s weapon, bind up the opponent’s weapon, or be thrown over their head to blind them. Italian duels were complicated, convoluted things, with many social rules and customs to follow. I recommend The Sixteenth Century Italian Duel by Frederick Bryson, from which I quote:
“The authorities were not in agreement as to which was the party by whom the arms should be chosen, or as to when his opponent should be told what they would be. Chiefly because of the former question, or because of objections to arms which had been proposed, the day appointed for the duel was often spent in disputes, which sent the spectators home disgusted. At some time before the middle of the sixteenth century, a time which Fausto da Longiano (1551) called the best period of the duel, the selection of not only the arms but also the dueling-field and the judge was prohibited to the party who had begun the quarrel. The writer said that one who had received a blow, however, ought to lose the choice of arms if he had replied by stabbing. “
It was customary for the challenged party to select from a list of possible arms, and he might choose the weapons which he was most proficient in or gave him an advantage of his opponent.
Until the mid-to-late 16th century, duels between noblemen were fought on horseback, using one or a combination of broad-sword, rapier, dagger, lance, shield, battle axe, mace, and pistol, however the axe and mace were not typically used because they were unwieldy.
The by early 17th century, duels were most often fought on foot. Bryson write “Except for the case that a duelist was physically weak, the combat on horseback had lost its popularity because men were unwilling that a victory should be shared with a horse”
The armor to be worn by both parties was also an issue for argument and clever weaseling. The custom of honor eventually needed to dictate that the armor chosen could not be too heavy for a weak opponent to use, nor could it obstruct vision to favor one party, as in the case of a helmet proposed by a one-eyed duelist, which covered one eye to make the opponent as half-blind as he was, thus giving the one-eyed man the advantage of having lived and adapted to the condition.
Bryson cites some fantastical tales by Brantome in chapter III:
For a certain duel between two gentlemen of Rome, said Brantôme, the armor extended from head to foot, except for an opening twice as large as the palm of the hand. This hole was just over the heart. The party who had the choice or arms practiced with a fencing-master and learned to strike always in the spot which was unprotected. One may imagine the result (Brantôme, VI, 295-96).
[...]
In Piedmont, said Brantôme, there was a duel between a young gentleman-soldier and a Gascon sergeant. The latter, although he was the party challenged, allowed the choice of arms to his opponent, who chose to fight with sword and dagger. Being small, moreover, he demanded that there be worn a steel collar with sharp points, which pre vented his taller opponent from looking down. Although the right to require the use of this equipment was disputed, it was finally allowed, and the small man easily killed his opponent (Brantôme, VI, 293-94).
The choice of weapons was similarly governed by rules – they could not be too large for one party to use, nor could they be required to be gilded or jeweled, such that a poorer dueler might go broke trying to acquire the specified weapon.
Also, some choice of weapons might be seen as too vulgar for a person of nobility, such as the use of clubs or wood-axes. However, Bryson makes mention of one authority who approved the use of slings and stone, declaring them as legitimate combat weapons since ancient soldiers had used them.
The vast majority of duels in the 16th and 17th centuries were fought with the traditional weapons gentlemen carried – the sword, the dagger, and sometimes the cloak. The use of pistols were challenged at the time, since they were seen as unreliable, a danger to spectators, and did not allow the combatants to display proper courage and virtue.
Although the weapons were traditional and generally normal, some people tried to get tricky with them: Another anecdote by Brantome, by way of Bryson:
In Giraldi's Ecatommiti there is an account of a duel in which each party was allowed a spear and a sword; and while one of the contestants according to the regular custom began to fight with the spear in his hand and the sword at his side, the other as he entered the field attached the sword to the spear, and with this long weapon he killed his opponent (Ecatommiti, Deca X, nov. 8)
And another:
“He mentioned also a case in which a man ordered to be made at Milan two swords and two daggers so tempered that unless handled in a certain manner they would break like glass. Then in a duel fought without armor the party who had been forced to accept these weapons and who, unlike his opponent, had not learned how to use them broke them and was killed by the other party. Brantôme noted that it was too late to object to a weapon after accepting it and going to the duel”
Although these are anecdotes (albeit from the time period), they demonstrate the clever ways people would twist the conditions of the duel to their advantage or to even the odds.
Source: The Sixteenth Century Italian Duel, Frederick Bryson, University of Chicago Press, 1938
http://historicalfencing.org/PDF/Bryson%20-%20The%20Sixteenth-Century%20Italian%20Duel.pdf
Edit: Formatting and typos