The Japanese royal family has the longest continuous reign on Earth, going back at least 1500 years and covering 125 monarchs. However, that doesn't necessarily mean there weren't any times when there was contention over who might be the next to ascend to the throne.
Google has failed me, largely because there is so much being said about the issue of female ascendency, and the current emporer's desire to abdicate, that it drowns out the results I hope for, assuming they even exist.
Has there ever been a situation in all of the Japanese royal family's history when the throne was contested between two potential heirs that claimed legitimacy? Is such a dispute even possible with the rules of ascension Japan has had previous to 1889?
Yes, there is an event called the Hogen Rebellion, and the events are chronicled in Hogen Monogatari.
To summarize the events, Emperor Toba retired in 1123; however, he still pulled the strings from behind the curtain, first through Emperor Sutoku (his son), and then when Sutoku abdicated, Emperor Konoe (Sutoku's younger brother).
However, when Emperor Konoe died without an heir in 1155, and Toba the year after, the court was thrown into turmoil. Eventually Konoe's mother and her allies within the Fujiwara family managed to install Go-Shirakawa (the middle child between Konoe and Sutoku) as emperor; this is contested by a rival court faction, who favours the retired emperor Sutoku.
Long story short, Shirakawa prevailed and Sutoku and his courtiers are sentenced with exile.
First you must consider that for many of those 1500 years, the "emperor" was mostly a religious figurehead, in no way the real ruler of Japan. His duties were essentially to preside a large number of religious ceremonies.
Also the imperial power (such as it was) was very often wielded by one or more retired emperors. What happened was that the titular emperor, tired of his ceremonial duties, did officially retire, often becoming a monk, and a young member of the imperial family was elevated to the throne.
The retired emperor, free of the official commitments, was then able to exert his influence more effectively. This practice was called the rule of the "Cloistered Emperor", and was very much the norm for long stretches of time, notably in the twelfth century with emperor Shirakawa and his successors.
That said, there was at least one long period where there were two competing ruling emperors: the time of the two courts, the Northern Court in Edo (Kyoto), in realty under the control of the Ashikaga shogunate, and the Southern Court in Yoshino, established by Emperor Go-Daigo in 1337 after he fled Edo when the Ashikaga forces occupied imperial capital.
The two imperial courts battled for more than fifty years until 1392, when the power of the Ashikaga administration forced the last southern court emperor to ask for peace and return to Edo, bringing with him the imperial treasures.