I know there's a difference between the magnetic and geographic poles, but when and how did explorers determine the point in Antarctica that was the south pole? Did they use the stars to guide themselves?
Good question!
Sextant and the sun.
The early explorers only traveled there during the summer months (24 hours of daylight), so they never had the stars to guide them. At the exact south pole, the sun will be at the same angle in the sky -- measured with a sextant -- no matter the time of day. Roald Amundsen knew that sextants have some error, so after they arrived at the pole he sent team members 20 miles in each direction to firmly establish the claim of being first.
Side note: a compass does work at the geographic south pole, it just points in a wildly wrong direction. To find the magnetic south pole, they used a kind of compass that points up-down called a dip magnetometer. When it was straight up and down, that was the magnetic south pole. For a really good story about that exploration, read Mawson's Will.
Edited: spelling