I'd assume there would be a huge language barrier. Did any Americans speak Japanese to communicate with the Japanese? Or did some diplomats in Japan speak English?
The diplomatic letters Perry wrote were translated into Chinese and Dutch by American diplomats who knew those languages. The actual verbal communication was handled by, for example, the Japanese negotiators speaking in their native language to their translator, who would translate it into Dutch for the Americans' translator, who would then relay the message to their side in English.
The Japanese had considerably increased their Dutch skills from the late 18th century onward as they were increasingly aware of the need for Western knowledge, and the Dutch at Dejima were the only real contacts they had with the West. Knowledge of Western languages other than Dutch in Japan was very low at that time.
Source: Jansen, Marius B. The Making of Modern Japan. Cambridge: Harvard University Press, 2000.