Settlement expanded in the British-occupied areas of Australia (really just New South Wales and Tasmania) from the 1820s. Having struggled through the early years, agriculture, sealing and such were expanding, with Old World class boundaries heavily damaged by the intermingling of free settlers, expired convicts and former servants of the Crown all vying to make the land pay.
By the 1830s there was a labour shortage, and convict transportation could not begin to make this up. Law in New South Wales was evolving: the lash and harsh discipline had underpinned the master-servant relationship in the early years, but this was problematic with many more workers now being freemen. Landowners questioned whether convict labour was actually efficient, while in England transportation was under scrutiny as an effective form of punishment (as more 'enlightened' ideas of reforming prisoners took effect). From 1838 a committee of the House of Commons under Charles Molesworth heard submissions from colonists and concluded Transportation should end.
In 1840, Transportation to NSW was abolished, and although this was quite acceptable to the authorities in NSW, it was imposed from England. Some objected to the changes but the colony was already developing a divided social structure with 'old' settlers (of fortune), free native-born colonists, and free arrivals all occupying their own strata with those close to their convict stock at the bottom. Despite this, locals were already 'level' in the absence of the trappings that underpinned the authority of the wealthy back home, such as large opulent houses. There were just too many signs of how equal people were.
Transportation to Van Diemen's Land (Tasmania) continued until 1853.
It should be noted that the population of NSW doubled between 1834 and 1841, with exports (built on wool) increasing almost ten-fold, and things like steam engines and gas lighting being introduced. It was a period of rapid change and development, and the 'convict' element (convicts or their children) in society went from being a majority in the early 1820s, to a small minority.
In the 1840s Melbourne was settled (taken from the indigenous population) and grew rapidly, ideas about the country as a worthy and prosperous place took root (as opposed to being the godforesaken harsh land of punishment), and society became dynamic, with social organisation centering on normal economic relationships.
The idea of Australia as a prison colony simply became obsolete.
Sources: History of Australia Manning Clark. Black War Colive Turnbull. Essays in Economic History of Australia James Griffin (ed).