In this photo, the HMS Dreadnought appears to have masts for sails.
So what's with the masts, were they just for show?
They are used for displaying signal flags. Wireless was still a new technology and not always reliable. And light signalling reauired uniterrupted line of sight.
So even today naval vessels use signal flags to convey orders or notices which can be lengthy chains or multiple at the same time. Though it could still lead to errors, most famously at the Battle of Dogger Bank.
That isn't a full sailing rig. If you want to see what full rig looks like on a battleship, here you go. Laid down in 1873, the HMS Shannon was the last full-rigged armored cruiser which was totally designed for sailing. While there was a propeller, it could be lifted from the water to reduce drag under sail. A few ships retained full rigging for a time after that - the HMS Inflexible, launched in 1876 for instance, had full rigging, but sails were essentially a joke at that point. It isn't an exaggeration to say that that even with every sail put out and the table-linens too, the ironclad Inflexible would barely have budged. The fact that these ships retained sails is really just a testament to the stodgy old dudes who didn't want to get rid of the tradition. Arguments such as having the alternative propulsion would save fuel were preposterous, since the additional weight of the masts and rigging required more fuel than the sails ever could save!
The HMS Ajax was launched laid-down in 1876, and was the first capital ship the Royal Navy designed from the start (A few earlier ones had dropped the sails before launch) lacking sails (and no sails aside, was a crappy design), but it would still be some years before the practice died totally, with the armored cruiser HMS Imperieuse, launched in 1883.
Of course this only partially answers your question! If those aren't for sailing, what are they for? Communication and Observation! Signal flags would be run up to communicate with other ships as you can see in the HMS Shannon picture (recall that radio was only just becoming viable then), and as you can see, there is an observation point in the mast.
So that's the sum of it. If you want to read more about this, I'd recommend Birth of the Battleship: British Capital Ship Design 1870-1881 by John Beeler,which is what this is all from, or perhaps /u/jschooltiger will show up and expand on this.
Edit: Ajax had her keel laid in '76, not her launch.
[Response to a deleted comment, but perhaps relevant]
The (then controversial) jettisoning of a full rig came after HMS Devastation in 1870, though some ships did revert (and then later were cut down when this was seen as pointless, especially in light of the HMS Captain disaster/capsizing). To ameliorate the possibility of running out of fuel, the Royal Navy (and a few others) took great pains to build collieries (really big piles of coal) on a variety of islands, piles that can still be seen; seaborne colliers also became widespread in the rare instance that a ship might run out of fuel. (In practice, ships usually carried a significant excess of coal--even when not being used, the idea was that a full bunker was another layer of protection. Coal dust explosions notwithstanding.)
Jon Beeler's The Birth of the Battleship is an interesting read on a lot of these developments, and doctrine in particular.
On a related note the IJN had pagoda masts on their battleships. A pagoda mast was more of a tower than a mast, it had searchlights, observers, and fire control.
