Or did they keep an oral history of where they came from?
The Rapanui (when speaking of the language and the people, it's one word; when speaking of the island, it's two) knew distant lands generically as "Hiva." Their ancestors had come from Hiva; Europeans were known as the Tangata Hiva ("People from Distant Lands"); the tiny islet of Salas y Gomez (~390 km east of Rapa Nui) was known as Motu Motiro Hiva ("the islet on the way to Hiva"). The Rapanui (back before they were the Rapanui - that's a name they adopted in the 1860s^† ) probably reached South America, were contacted by South Americans, or both. Not only did they know that here was land beyond Motu Moitro Hiva, but they possessed sweet potatoes - a South American crop which was eventually spread throughout Polynesia and for which the Polynesians adopted the Quechuan word for the plant (variations on kumar(a)). Additionally, evidence for Pre-Columbian gene flow between South American populations and the Rapanui has recently been uncovered.
Historically and archaeologically, the Rapanui have influences from both the Marquesas (where Hiva is a common place name) and Mangareva (whose legendary founder Atu Motua bears a striking resemblance to his Rapanui equivalent, Hotu Matu'a). The exact nature and sequence of these influences is debated, with the Marquesas and Mangareva Islands have their own complicated history of interaction.
Pre-colonial influences aside, the Mangarevans are known to have had recent historical influence on the Rapanui, in the form of Mangarevan missionaries that arrived on the island in 1866 as part of the Congregation of the Sacred Heart mission.
† Rapa Nui got its name after a blackbirding expedition (essentially slave traders in all but name) that had captured several people from the island, including Manu Rangi, the eight-year old ariki mau (the "king" of the island, though the political power of the ariki mau had diminished considerably in recent generations) had been liberated by the people of Rapa. In the Rapan language "Rapa" means "Extremity," and the island was named such because it its great distance for other islands. Once the ship had been liberated, the Rapa compared geographic knowledge with the people they rescued and determined that their island wasn't as far away as others in comparison. They changed the name to Rapa Iti, "the Lesser Extremity" and gave the other the name "Rapa Nui," the Greater Extremity." The Rapaiti and Rapanui sailed the liberated vessel first to Tahiti, then back to Rapa Nui, where the new name became popular.