I can answer this for the Conservative side. The label 'Tory' did not come into regular usage until 1827 and was used by opposition Whigs to criticise not mainstream, governing Pittites but the backbench 'ultra-Tories' (a misnomer, since they had nothing in common with the original Tories of Queen Anne's reign and were in fact ultra-Protestants seeking to preserve the Whig Revolution settlement of 1689). It was not until the crisis over reform in 1831 that Pittites began referring to themselves without qualification as Tories. However at the same time Peel preferred the name 'Conservative' and in 1833 the first Dod's Parliamentary Companion divided MPs into Conservatives, Radicals and Liberals.

'Conservative' became popular as it conveyed how, with the collapse of the Anglican regime in 1828-1832, the Pittites transformed themselves into the party of order, seeking to preserve property.

Source: Boyd Hilton, A Mad, Bad and Dangerous People? England 1783-1846 (Oxford: Clarendon Press, 2006).