In the film The Witch, the two parent characters seem to have considerable anxiety that their pre-adolescent son was lusting after his older sister. Was this a common anxiety among Puritan families, especially because of their seemingly large family sizes?

This would not have been indicative of puritian societies. In this specific case it seems the brother was attracted to his sister, having just entered puberty and having no other female that close to his Age around it makes sense he would direct feelings toward her. As for the parents they thought the brother being attracted to his sister was a result of them beliving she was a witch who cast a spell on him.