As /u/The740 summarizes in this post, (I assume he is correct, if not I'd be interested in a rebuttal) The Puritans were not actually the sexually repressed, conservative fanatics that we often see them as in pop-culture. So why/how did they get that reputation?
Firstly, what /u/The740 says is accurate (I've posted similar comments in the past).
The stereotype itself is fairly old - Bruce C. Daniels in Did the Puritans Have Fun? Leisure, Recreation and the Concept of Pleasure in Early New England writes that "'puritan' has been a synonym for dour, joyless, repressed behaviour" for over four centuries.
That said, a large portion of the modern (mis)understanding of the puritans comes from Nathaniel Hawthorne's The Scarlet Letter, in which puritans are depicted as opposed to all happiness and leisure. This idea took hold and wasn't really questioned academically for the next hundred or so years.
H.L. Mencken - who famously quipped that Puritanism was "the haunting fear that someone, somewhere, may be happy" - also deserves some of the blame as he pointed to the puritans as those responsible for the "Victorian America" that he so derided. He used "puritan" as a pejorative to describe those he didn't agree with, and it largely stuck.