Did they know the earth was round and did they ever explore in that direction?
Japan underwent strong Chinese influence in the 600s after Prince Shoutoku brought Buddhism and by extension a large amount of Chinese thinking to Japan. China at the time, as part of its "Middle Kingdom" status, defined the earth as a "square" and heaven as surrounding it. It was also around this time that the name "Nippon" started appearing, or the land of the Rising Sun.
Japan never developed ships that were large or sophisticated enough to travel the Pacific until after Europeans arrived. I am not aware of any expeditions that were sent out, but one must keep in mind that Japan is a relatively large country. As far as many people were concerned, distant parts of Japan were basically another country. The origin of the Shogun, for example, came from a Heian-era expedition to subjugate the Emishi in what is now the Tohoku region on the main island of Honshu. Hokkaido, the most eastern part of Japan, had only tentative settlements in the 1400s, and was only fully annexed around the Meiji Restoration. Basically, Japan did explore to the east-just not across the Pacific.
Source:
Mason, A History of Japan
Werner, Myths and Legends of China
Also, a visit to Nara, which has several exhibits on Prince Shoutoku.
Before the Europeans arrived in Japan in the 16th century, the Japanese thought the world existed of Japan, China and India. They had not had contact with any other regions, so their knowledge of the rest of the world was understandably quite limited.
Oda Nobunaga, a Japanese leader at the time of the first contacts with Europeans, showed considerable interest in geography. He showed great interest in the geography of India and a globe of the world brought by the Westerners, who recorded the conversations.
Source: Geography and Japanese Knowledge of World Geography by Shintaro Ayusawa
Not exactly the same question, but I previously asked about how the Japanese came to know about the Americas, so you might find something interesting in that thread: http://www.reddit.com/r/AskHistorians/comments/17di7o/when_and_how_did_the_japanese_chinese_and_other/
As a corallary what about the lands to the North?
Follow up--how far west were they aware of? Did they know of anything beyond China?
I'm surprised that no one has mentioned this, but, while everyone knows Zen, the most popular form of Buddhism in Japan has been, for centuries, Pure Land Buddhism. One of the ideas is that there are various "pure lands" ruled over by various buddhas. To put it in a Western metaphor, it's basically impossible to sin/commit karmic no-nos in the pure lands, so if you're reborn there, it's like an easy pass to moksha (release from the cycle of rebirths). Pure Land Buddhism concentrates on appealing to the Amidha Buddha who is the Buddha of the Western Pure Land. However, I believe there are pure lands at all the cardinal directions associated with each of the Wisdom Buddhas. Akshobhya is apparently the Buddha of the Eastern Pure Land. Now, the idea of the Eastern Pure Land is one that is essentially as old as the Western Pure Land, though it was never as popular. Now, the question is, was this the literal belief of the people? That if they went far enough, they'd come to Akshobhya realm? I don't know. But I did find an article suggesting Japanese monks would occasionally engage in a practice called fudaraku tokai, in which they'd "set out to sea in a small, single-sailed (but oar-less and rudderless) boat in the hope of arriving at the southern paradise of the Bodhisattva of Compassion (Avalokiteśvara; Guanyin; Kannon)." While the author argues that this is not some purely Buddhist practice, of course nothing in Japanese religion is purely anything, and it's suggestive that at least some people would have assumed that Akshobhya's Eastern Pure Land was out there, somewhere due East of Japan.