Quoting myself from an answer to a similar question:

A note about the puritans: we have the stereotype of the puritans as anti-sex, but they actually rebelled against the Catholic Church's teachings that all sex (including marital sex) was sinful to some degree (even if just because of the passions and resultant pleasure). The puritans felt that sex was an important part of married life, and not just for procreation.

Leland Ryken in Worldly Saints: The Puritans As They Really Were writes: "when a New England wife complained, first to her pastor, and then to the whole congregation, that her husband was neglecting their sex life, the church proceeded to excommunicate the man."

William Gouge, a puritan preacher, said that married couples should engage in sex "with good will and delight, willingly, readily, and cheerfully."

Further, the large number of puritans who had their first child less than nine months after getting married shows that the puritans were definitely having sex outside of marriage too.

As for the stereotype itself, the modern (mis)understanding of the puritans comes largely from Nathaniel Hawthorne's The Scarlet Letter, in which puritans are depicted as opposed to all happiness and leisure. This idea took hold and wasn't really questioned academically for the next hundred or so years.

H.L. Mencken - who famously quipped that Puritanism was "the haunting fear that someone, somewhere, may be happy" - also deserves some of the blame as he pointed to the puritans as those responsible for the "Victorian America" that he so derided. He used "puritan" as a pejorative to describe those he didn't agree with, and it largely stuck.

Further, /u/plusroyaliste has an excellent post about the Puritans generally, and this thread more closely deals with your question specifically.

This may also be a good question from /r/askanthropology

Additionally, how much of this can be attributed to the Second and Third Great Awakenings during the 19th century?