To understand the Puritans in America you need to understand the place of the Puritans in England, and the way Europeans in general in the 17th c. were religious.

Whether you were English or French or German, you would think that there was one denomination that was correct, that was The Church, and that everyone should belong to it. There were some groups that felt there should be "liberty of conscience": that everyone should follow their own hearts: but these ( Anabaptists and Brownists) were thought very radical and suffered real persecution.

The Anglican Church in England was the established church there, after Henry VIII had taken England out of the Catholic Church. But although the Anglican Church had left the Catholic hierarchy, was not under the control of the Pope, the division between its beliefs and Catholic theology was not that distinct. Most importantly, it was not Calvinist: it had bishops and archbishops, did not agree with Calvinist doctrines like pre-destination. The Puritans were a group within the Anglican Church that wanted to reform it to be more Calvinist. But they ran up against very strong opposition from the King ( or Queen), and the head of the Anglican Church, the Archbishop of Canterbury, often went to great lengths to suppress them.

The Puritan-established colonies in North America, like the Boston Bay Colony, lacked that Anglican hierarchy and were pretty much beneath the notice of the King. They were able to implement the reforms they hoped to impose upon the church in England without fear. However, without that fear of persecution and suppression, divisions over doctrines and theology within the colonies could easily grow. People like Roger Williams and Anne Bradstreet, for example, thought that it was possible to tell who was in a state of grace, i.e. was pre-destined to be saved. The colony tried to suppress them, much as the Anglican church had suppressed Puritans. Bradford and Williams were banished. But soon there was a variety of colonies with a variety of religious beliefs, as more colonies were founded, and religious suppression simply became impossible. Virginia would be strongly Anglican, and Maryland would be Catholic, Pennsylvania would be founded by a Quaker. Yet German Lutherans would immigrate to Virginia, Anglicans to Maryland, Scotch-Irish Presbyterians to Pennsylvania, and Roger Williams would make Rhode Island welcoming to everybody, the proverbial "home for the otherwise minded". The Puritans no longer were in the Anglican Church trying to reform it, and it was impossible for them to exert their own control over all the colonies. For the most part they gradually came to a general consensus on what their own church should be, which became the Congregationalist Church. That was essentially the Massachusetts established church- for a time.. But in the latter part of the 17th c. much of the reforming zeal of the original Puritans greatly diminished, and other denominations were able to establish their own churches even in Boston. By the mid 18th c. the typical New England town commons would have a Congregationalist church, and likely an Episcopal Church as well.

Morgan, Edmund S. (1958) The Puritan Dilemma