The fictional Captain Jack Aubrey, RN, captures a slave ship and is so appalled by the conditions he finds that he chains the captain and officers to the ship and then has his squadron use it for gunnery practice until it sinks. Were the Royal Navy very harsh with slave traders they captured?

As the OP says, in this case, there's some of O'Brian's usual work to reference the historical backdrop of the Napoleonic Wars.

In this case, Aubrey's first mission to interdict the slave trade is actually chronologically appropriate. The novel The Commodore takes place in 1812, five years after Great Britain had abolished the Atlantic slave trade in 1807. The United Kingdom would not abolish slavery in its colonial possessions until 1833.

From 1808 onward, the Royal Navy was empowered to overtake and capture ships in the Atlantic with slaves on board, returning any slaves captured in the process to West Africa, most frequently Freetown in Sierra Leone but also as the 19th Century went on, to St. Helena, Cape Town, Lagos and other British-controlled ports in West Africa.

Other Atlantic powers abolished the Atlantic slave trade within a few years of the British decision--the United States also in 1807, Portugal in 1810, France in 1814. This created a complicated series of treaty agreements covering the activities of the "anti-slavery squadron" of the Royal Navy over the course of the 19th Century, which sometimes kept British naval officers from acting against a particular ship. Just to give a sense of how complicated it all was, Portugal's commitments on the slave trade in 1810 allowed Portuguese ships to trade in slaves from Portuguese-controlled outposts, just not in outposts controlled by other European nations or in territories controlled by West African sovereigns. When Spain signed a treaty on the slave trade, it only committed to abolition north of the equator. Essentially the British most of the time could only interfere with a ship that had slaves already on board--ships that were equipped for the trade but with no cargo had to be left alone. Captains involved in the Atlantic slave trade also looked for ways to avoid the Royal Navy's activities either by having faster ships ("Baltimore clippers") or by procuring slaves in other locations than the dominant West and Central African slaving ports of the late 18th Century, including a brief period of intense slave-trading activity in East Africa.

In the period of time that this novel takes place (keeping in mind how fluid time sometimes is in the Aubrey-Maturin books), Aubrey would have been interdicting slavers before some of the treaty framework was fully established, and just slightly before the first significant operations of the Royal Navy, but after the 1807 and 1810 agreements. So while Aubrey's operations in the novel are a bit bigger in scope than is historically accurate for that precise time, it's still within shouting distance--there was already a commander of the West Africa Squadron in 1808.

Aubrey's attitudes as described by the OP are essentially necessary for the characterization established by O'Brian. Aubrey and Maturin are meant to be basically benevolent and palatable to modern audiences--they have more "period" personality than a purely cartoonish figure like Horatio Hornblower and more roundedness, but if Aubrey remained indifferent to enslavement in the course of his duties in this novel, it would make contemporary readers fairly uncomfortable. So Aubrey's reactions are rather histrionic and exaggerated and yet not strongly implausible. (Maturin's abolitionism is consistent with his character and very much in line with sentiments that existed in 1812.)

Sentiment about slavery among the British elite and groups who were at the edges of their social world (e.g., people like Aubrey and Maturin) was shifting rapidly in the early 19th Century. Olaudah Equiano (aka Gustavus Vassa) published his autobiography of his enslavement and manumission in England in 1789--it was a gigantic bestseller in the UK. In the 1790s, attempts to abolish the slave trade were soundly defeated in Parliament, but William Wilberforce's continued efforts for abolition won more and more support in the lead-up to the passage of the 1807 Act. (Nelson's devotees claim he had no particular views of slavery, but he was friendly with plantation owners and was at least once very dismissive of Wilberforce and his allies.) By the 1820s-1830s, moral opposition to the slave trade was fairly established within polite society in Britain and was an important part of Nonconformist religious evangelism.

What is more interesting is that the last twenty years of historical scholarship on maritime life in the late 18th and 19th Century Atlantic has made clear that many Atlantic crews--including Royal Navy crews--were multiracial and included people who had escaped from slavery as well as sailors who had participated in slave voyages and had come to feel great opposition to the trade subsequently. Peter Linebaugh and Marcus Rediker's pathbreaking The Many-Headed Hydra is a terrific window into this maritime world. (O'Brian's books were written substantially before much of this new research, so his characterization of maritime life doesn't fully tap into this new historical picture.) So you could imagine a slightly different version of Aubrey's expedition where some of his reaction to slavery followed from being a part of that milieu. Moreover, by the early 19th Century, when the Atlantic slave trade had been at its most intense peak for several decades, slave ships were built in highly specialized ways to contain as many slaves as possible in profoundly miserable conditions and were also fortified on deck against possible slave revolts. (Rediker's The Slave Ship provides a horrifying and depressing look at how slave ships actually operated.) It's fairly consistent with Aubrey's attitude towards naval service and maritime life that he might find his first actual up-close inspection of a slaving ship deeply revolting and a perversion of life at sea.

However, because of treaty obligations, in general Royal Navy officers had to be careful to follow legal guidelines in inspecting slave ships and taking them as prizes--eventually there were multiple naval courts designed to ensure that everything was done properly. So Aubrey sinking a slave ship out of rage is a bit Hollywood, yes.