I have heard of this idea before and it really isn't substantiated. More so when one considers actual Inca stonecutting practices. The way they cut stone from the bedrock, shaped it, and fitted the stones together to make a wall is a lot less impressive than one thinks. Besides using natural stones already near the construction site to get those large boulders, a lot of smaller blocks were brought in as well. Furthermore, the face of a stone wall is a facade that fools people into thinking that the blocks are closely fitted throughout the wall. In reality, only the first few inches are well shaped to fit snugly together. Past that point there are large gaps that are filled with dirt and rubble to provide support for the wall. Inca stone cutting is actually a really lazy process when one looks at it. The irregular shapes of the block are because the masons were trying to use as much of the natural shape of the stone as possible with minimal dressing.

I did a write up on /r/badhistory not too long about Inca stonecutting if you are interested.

The articles I cite are

• Protzen, Jean-Pierre. 1983. Inca Quarrying and Stonecutting. Ñawpa Pacha: Journal of Andean Archaeology 21:183-214

• Protzen, Jean Pierre. 1985. Inca Quarrying and Stonecutting. Journal of the Society of Architectural Historians 44(2):161-182.

Both of them have Dropbox links in the /r/badhistory post.

Something I've always wondered when people talk about how tightly fitted the blocks of the pyramids and other structures are is the effects that earthquakes might have over millennia. I suppose, with less weight the tops of these structures would be less tightly fitted. Do you know if this has been considered?