Those were big and important landspaces ripe for the taking. The West wouldn't have been able to defend them and Japan would control half of the British Commonwealth right now.
The reason why the Japanese did not seriously contemplate an invasion of either Australia or New Zealand was both territories was because of their geographic size. The main priority for the Japanese was to occupy and secure what Japanese planners termed the "Southern Resources Area," essentially Southeast Asia, Netherlands East Indies/Indonesia, New Guinea, and the Philippines. The occupation of these areas stretched Japanese logistical forces to their breaking point. According to John Parshall's analysis of a hypothetical invasion of Hawaii, the Japanese has a sealift capacity for about eleven division in late 1941. Attacking a large and hardened target like Hawaii, let alone Australia or New Zealand, was simply out of the question unless Japan abandoned its drive for the Southern Resources Area. Any question of abandoning those operations was a dead letter since that was the basic strategic rationale Japan used to justify entering into the war. There was some tentative floating of planning for an invasion by the IJN during the heady days of early victories, but most of this was impractical and vetoed by the IJA and Prime Minister Tojo Hideaki.
This does not mean that the Japanese ignored Australia of New Zealand. On 19 February 1942, four of the aircraft carriers that raided Pearl Harbor attacked the northern Australian port of Darwin as part of an operation to secure Japan's southern flank. The raid consisted of 186 carrier aircraft and was coordinated with 54 land-based bombers operating from Celebes and the Netherlands East Indies. The air raid was a spectacular success, sinking eleven ships and damaging Darwin's port facilities. Japanese aircraft based on East Timor would periodically raid northern Australia's coast in 1942 and 1943, although never on the scale of the February attack. In addition to an rather ineffectual submarine blockade of Australasia's sealanes, on 31 May/1 June, the IJN conducted a midget submarine attack upon Sydney. Like most midget submarine attacks, the Sydney attack did not live up to the expectations of IJN planners, sinking only the ferry HMAS Kuttabul despite the presence of several warships in the harbor.
So while Japanese planners were aware of the vulnerability of Australia, conducting military operations beyond these temporary pinpricks was simply beyond the reach of Japanese military capabilities.
Except they werent. They were sparsely populated land masses without the oil, and natural resources that Japan sought, and were huge and so far away as to be impractical to occupy. Even during the war the Navy was dead set against any sort of invasion. It is very likely that any invasion of the Australian mainland would have just devolved into guerrilla warfare with the Japanese Army being run ragged across the continent.
The vast majority of Commonwealth strength in the Far East was in Singapore, between the substantial garrison and it serving as home port to the Royal Navy.
But it is a moot point as AUS/NZ did not represent the strategic goals for Japan, those were the East Indies and their resources, which lay on the other side of the US protectorate in the Philippines. It was felt that any further Japanese expansion would be met in kind by US escalation towards hostility, thus the expected war seemed ever more likely and Yamamoto developed his strategy to knockout the US Pacific Fleet and allow Japan freedom of movement to gobble up what she wanted.