As a secondary question, I'm curious about the differences between India and Japan in particular.

Both had large cities and armies as well as commerce and trade with European powers including the sale of weapons. Was Japan being united under the Shoguns, even nominally, a major factor as well? The British certainly heavily relied upon exploiting conflicts between Indian rival kings to expand their influence there, was this less easily possible in Japan?

/u/liqkerin

But they did. Nagasaki was a major post accessible to the Portuguese, used for a very important Nanban trade when Japan could not or would not trade with China directly. Tokugawa Ieyasu actively sought trading contracts with the Portuguese and in particular the Spanish in Manila. However, over time as Tokugawa became shogun and due to his suspicion of the foreigners he banned the Portuguese. I've written a bit on this here. Their rivals the Dutch accepted into a highly restricted trade in Dejima island nearby. This kind of arrangement was special, but not unique. In Macau, the Portuguese accepted theoretical suzerainty and paid tribute to the Chinese emperor.

If you are to compare with Europeans in Batavia, Malacca, Goa, Manila et al., it all depended on what it took for them to get what they wanted, namely access to goods and superiority in trade under their terms. European activity was confined to their trading posts and bases. It is a mistake to say they colonized, especially not in the sense it is understood in the 20th century. The Dutch VOC was willing to pay tribute to Mataram as the Portuguese was willing to pay tribute to China.

There was a lot of collaboration and accommodation between various powers: European, Asians, local and foreign.

They tried. The Japanese substantially limited contact with the outside world, especially European powers, during the Edo period, starting from the 17th century. European powers challenged this situation from time to time, but in general they lacked the power and the will to defy the Japanese government. There was limited trade with Japan, primarily through the Dutch, at Nagasaki. In the 19th century this situation came to a head with several incidents of European powers either strong arming the Japanese or attempting to do so. The Japanese instituted a "shoot on first sight" policy (the "ninen nashi" or "no second thought" law) in 1825 that directed coastal forces to arrest or kill anyone attempting to land on Japanese territory. In 1830 Americans settled the Bonin Islands, which had been claimed by the Japanese though unsettled. After China was defeated in the First Opium War Japan changed its policy against foreigners, allowing them to be refueled and reprovisioned at Japanese ports. Additional attempts to open up trade with Japan were still rebuffed however, until 1853 when Commodor Perry brought four warships and forcefully negotiated an opening of trade.

Afterward the old government fairly rapidly collapsed, to be replaced by a very different government (the Meiji restoration). The Meiji government was fairly open with regard to relations with foreign powers, so there wasn't the same situation as there was in China to try to take advantage of a weak but nominally uncooperative regime versus a strong and cooperative regime in Japan.

Japan basically experienced a civil war in the 1870s (the Satsuma Rebellion), which not only caused them to significantly beef up its military (just as the US Civil war had for America) but showed the world what the Japanese were capable of. Few powers were thus eager to tangle with the Japanese except for very good reasons. Japan then rapidly industrialized and grew stronger militarily through the remainder of the 19th century. They won a war against China, annexed Korea, and won a ground and naval war against Russia.

As it turned out, the window of time between when European powers had the wherewithal to forcibly open up Japan and force them to abide by unequal treaties before they became too strong to tussle with was only a matter of a few decades. During which nobody pursued the opportunity to pursue a colonization effort similar to what had gone on in India or China earlier. European powers had the ability to force Japan to open up, but they didn't have the ability to force Japan into outright subservience.

In short, Japan was simply a more powerful, and more centrally controlled, nation than those the European powers had brought to heel beforehand in service of colonial expansion, and one that gained strength rapidly during the period that Europeans might have otherwise attempted to pursue such colonization dreams.