More often than not you'll find veterans describing those hand-to-hand combat as "one of the few times the bayonet was actually used to deadly effect," or with words along those lines. The implication is that hand-to-hand combat was rare. There was also a hospital study conducted in 1870 that noted only very few bayonet wounds.

There is a counter-argument to this from Civil War News:

The 1870 Surgeon General’s Medical and Surgical History of the War of Rebellion (1861-1865) on Civil War injuries contained a table that listed the type(s) of wounds treated in Federal hospitals. Fewer than 1,000 bayonet wounds were noted there.

Connecting the dots, historians concluded that the bayonet was little but an impediment to soldiers who were issued them (not all were), used primarily as candlestick holders or cooking implements as noted in the “myth” heading.

This is not necessarily a conclusion that can be drawn. The Surgeon General’s information can be interpreted another way: only the wounded were treated in hospitals and the casualties from bayonets were primarily dead on the battlefield.

There is some weight to the argument since the weapon was primarily used in hand-to-hand fighting. Such close-quarter fighting provides a ratio of mortal bayonet wounds to recoverable injuries which is expectedly very high indeed.

Some period accounts state that few bayoneted soldiers survived the trauma due to the heavy loss of blood that resulted in such a short time. Perhaps if there were cause of death or autopsy reports from burial details we would have a different perspective of the damage done by the bayonet, but none are known.

The prevailing scholarly consensus is that hand-to-hand combat did not happen very often. The bayonet and the charge were used to dislodge men, but one side or the other would run away quickly.