One piece of the puzzle is that the climate on the East Coast is broadly better than the climate on the West Coast. Here's annual rainfall for the country. Notice that almost all of Australia's big cities are in the few regions that are clustered in the few relatively rain heavy regions of the country. When we also factor in rivers, again the East looks much better and more suitable for the agriculture that was the basis of the early Australian economy.

The second issue is that Australia was not discovered by Britain by sailing West to East, but rather East to West. Captain Cook was the first Brit in Australia, and sailed there via rounding Tierra del Fuego (in South America), not the Cape of Good Hope (in Africa). Captain Cook first landed in Botany Bay and, unsurprisingly, that's also where the First Fleet landed less than two decades later. They eventually built their first permanent settlement, what became Sydney, slightly to the north of Botany Bay, but still very much on the East Coast. Why? At this point the West Coast had been mostly unexplored by the Britis (though it had been explored by other Europeans, including the Dutch). Most of the other early colonization attempts were similarly on the East or the South. Britain didn't even claim the whole of the Australian continent until 1826, when they finally set up Albany not-quite-on the West Coast of Australia. Before that point, they had only claimed up to the 129th meridian (which still is the border of the modern state of Western Australia). This gif of Australian territorial changes might give you some sense of the early colonization.

There is a constellation of factors. First, the East coast has a more habitable climate and better agricultural land, more rivers and more rainfall. Then the East coast also has more natural harbours suitable for sailing vessels (a handful) than the West (none that I can think of). The East provides naval bases in the South Pacific and control of whaling and fishing grounds. The there is international law. Famously, the Pope had divided the world between Portugal and Spain along the 129th meridian. The dividing line runs along the border between the state Western Australia and the rest of the country (actually, a couple of kilometres East - the border was moved). East of that line was gifted to Spain, Britain's enemy, so the British thought it was up for grabs. The Western part was claimable by Portugal, a friendly nation, so off limits. When Portugal's power had declined, following the Napoleonic wars, the British felt it was safe to grab that part. For a controversial take see Collingridge's "The First Discovery of Australia and New Guinea" ISBN 0 330 27038 9 and McIntyre's "The Secret Discovery of Australia" ISBN 0 330 27033 8 which both suggest that Cook's strange behaviour suggested secret knowledge of the East coast. Behaviour like sailing past obvious safe anchorages like Sale, Jervis Bay, George's River only to stand off the narrow entrance into Botany Bay, waiting for the tide, almost as though he knew it could be a bit tricky, before he landed. Then sailing North, past more safe anchorages: Sydney Harbour, Broken Bay, Hunter River (Newcastle), Hastings River (Yamba), Moreton Bay only to run aground as he tried to go into the dangerous Round Hill Creek, near the town of 1770. But, for these see the sources mentioned.