They used celestial navigation, a very archaic form of a GPS. More often than not travels were between well known islands, only rarely were New locations discovered. With regards to Polynesians getting lost at sea, we can assume so, however, I am unaware of any polynesian settlements outside of this triangle so it a unlikely they had to travel too far and getting lost would be a rare occurrence.

While you're waiting for fresh responses, you might like to know that some of the material gathered in the FAQ under Polynesian culture & navigation addresses the questions you've posed in considerable detail.