Not to disagree with my fellow military history-flaired colleague, but there were at least some preliminary plans drawn up by the Imperial Japanese Navy to invade Northern Australia. The IJ Army disagreed with the plans (as they tended to do with anything Navy, and vice-versa).
The idea was to use the area of Darwin as a "area denial base," making it much harder for the Allied navy to operate either out of Australia or in the waters around. Most of the disagreements between the Navy and Army centered around manpower required. The Navy claimed three divisions would be enough, the Army thought ten or more were the bare minimum. In the end, the plans were shelved, to be revisited after South-East Asia had fallen and the Americans neutralized. Then came the Battles of the Coral Sea and Midway, and the plans never came back off the shelf. There was never any grand plan to invade the entire continent.
But to truly answer your question, "how close did Australia come to Japanese invasion in WWII," the answer is simple:
They did.
In January 1944, a fishing boat carrying an IJN Lieutenant and three non-coms landed in NW Australia to investigate rumors of a huge Allied military base being built there (false, as it turned out). The officers were there overnight before returning to their base in Timor. One of the officers later suggested a guerrilla campaign could be launched in Australia, but nothing came of it.
So there you go... some plans were made but not carried out, but a (very) small invasion occurred in 1944.
edit: changed tense.
Short answer is no.
With the US Navy at large, especially after Coral Sea and Midway, the IJN didn't have the resources to attempt an invasion of oz, which would have been a PRODIGIOUS undertaking!
Manpower was split between China, Burma/India, Philippines, the DEI, and the various islands and territories the Japanese occupied. Also an issue was the dogged Australian defence put up in New Guinea, later to be MASSIVELY augmented by American forces.
The distances to cover in Australia were too great, they would have NO, repeat, NO support from the local population, Australians or Aborigines. Throw in such obstacles as the Great Barrier Reef, and the presence of American and Australian airpower and seapower, such as in the MASSACRE of Japanese shipping in the Bismarck Sea, fantasies of Japanese boots on the ground in Australia were little more than that, fantasies.