It took a lot of patience, and labor, but the Inca stonemasons simply chipped away at their quarry stones with stones of equal or slightly higher hardness. This admittedly tedious effort also contributes to the precision with which stoneworkers were able to cut the rock. Settling a stone in its desired place and joining others was simply a matter of "okay, this rock is pretty close to the right shape, let me hit it in these places a few times, okay that's a bit more snug but this nub here needs to go..." and so forth. Stones like the "Twelve-Sided Stone" were not pre-cut to fit that form, but were more likely shaped into that form once it was put in place.

As I've said before, people don't give enough credit to banging rocks with other rocks.

Now, with regard to the sheer size of some rocks and fortifications, it is still a wonder to archaeologists and historians. That said, with enough labor involved one could move these massive boulders. The real trick is overcoming static friction in order to keep the stone moving, and as some experimental archaeology has shown, with enough folks that can be done pretty well. At that point it's a logistical matter of currying and caring for all that labor - and that was something for which Inca infrastructure was designed. Storehouses, highways, and a strong bureaucratic system could get together and support thousands of people for labor service to the empire.

An interesting anecdote I heard specifically about Sachsayhuaman (the fortification panorama I linked to above): recently some geochemical analysis was done on the stones there, and some of them appear to have come from far to the northwest, miles away. I'll have to ask my friend about it, but if this is true then some people really pissed off the Inca to have to pull a stone dozens of miles through the mountains for a fortress in the center of the empire. Considering that the stone came from the northwest, where several staunch enemies of the Inca once lived, this isn't unreasonable.