Rhode Islander here. I can really only attest to the history of my state. Founded by Roger Williams. He left Massachusetts in persuit of religious freedom and purchased the land known today as "The state of Rhode Island and Providence plantations" from the natives. I would imagine that the amount of land being purchased by 1 individual would have its limits.
A large part of this comes from how they expanded. All these colonies started primarily as coastal endeavours. Here is a map of the 13 colonies, with the fall line included. This line essentially served as a cut off point for easy expansion, since along it is where many rivers' easy navigability ended, and represented the limit of expansion for most early European settlers. If you exclude the area of the states past this line, the colonies are all very similar in size. http://mrkash.com/activities/images/englishcolonies.jpg
You'll also notice that this map does not show the final extent of the states that became, and this is because for both of your two main examples of larger states, significant expansion occurred post-independence as they expanded inland. This was actually a cause of a great deal of conflict between the states, even before independence. States like Deleware or Maryland, bounded on all sides by other colonies, did not have room to expand anymore, while states like New York and Penn. could.
The short answer is that the colonies were the result of various, often wildly different processes of creation and evolution. Colonies were founded by multiple countries and even within a single country like England there were many ways colonies were formed and defined. Most changed their form of government (or had it changed for them) multiple times. Most struggled with one another for control of territory. Most had expansive westward claims, but some did not. The reason each ended up the way it did by 1776 is nothing less than a century and a half of complex history.
So, for example, New York and Delaware and the colonies between them like New Jersey and Pennsylvania have a complicated history. The Hudson River was colonized by the Dutch and the Delaware River by Sweden. New Sweden was conquered by the Dutch in 1655, becoming part of New Netherland. The history of New Netherland is itself complicated but its claims reached from what is now Connecticut to Maryland. Actual colonization was mainly limited to the lower Hudson and Delaware Rivers by the time England took over in the 1670s. The Dutch called the Hudson "North River" and the Delaware "South River". Both were early non-English colonial areas.
After the English took over New Netherlands the region between the Hudson and Delaware (New Jersey) was split off. Basically, the Duke of York, to whom King Charles II had granted New Netherland, gave New Jersey to his friends Lord Berkeley and Sir George Carteret (Charles II's grant also included Massachusetts—he had a knack for granting land that had already been granted). This severed New York from the Delaware settlements. There was no particular plan for developing New Jersey. Settlers from various places infiltrated the region, mainly Puritans in the north and Swedes in the south. Before long New Jersey was split in two. The western and southern part was sold to Quaker proprietors who immediately began a vigorous colonization program.
One of these Quaker proprietors of New Jersey was William Penn, who went on to receive the huge land grant that would become Pennsylvania. The king owed Penn's father gobs and gobs of money. Penn accepted this huge land grant in exchange for the debt. Since the grant was for the lands west of the Delaware River it effectively hemmed in New Jersey. Originally Pennsylvania was tightly linked with the older New Sweden (Delaware) settlements, and William Penn considered what is now the state of Delaware part of his grant. But these "lower counties" protested and challenged the claim and gained a kind of semi-independence from Pennsylvania, which in time became full independence. Basically the settlers of old New Sweden were protective of their differences from the Quakers and their earlier independence.
So Delaware survived, but was small. New Hampshire survived in a similar way—it was almost absorbed by a powerful and aggressively expansive neighbor (Massachusetts). As for why Puritan New England fragmented while Pennsylvania, Virginia, New York, etc, remained large is a different story. But even though New England broke into parts it did "expand aggressively". Puritans founded towns far and wide, and well outside what is now considered New England (Newark, NJ, for example). Both Massachusetts and Connecticut claimed their territory extended west to the Pacific. They fought hard to keep their western claims and in the end demanded compensation. I won't get into why New England fragmented—that's a whole other long and complicated history.
Western claims are a major reason why some colonies ended up larger than others. Some colonies, like Delaware and New Hampshire (and to a lesser extend New Jersey) were lucky to survive at all and never really had westward claims. The New England colonies had westward claims (well at least MA and CT did) and the drive to expand and grow, but fragmented early on and ended up blocked in the west by New York. Unlike most other colonies, whose westward claims came from their original grants and charters, New York's western claims came from their treaties with the Iroquois and other Native Americans. New York ended up with vast western claims (most of which were eventually ceded to the US federal government) mainly because of their old and strong alliance with the Iroquois, the overwhelming power of the Iroquois in the Great Lakes and Ohio Valley region, and the importance of that alliance in Britain's long struggle with the French.
All of this only just touches the surface of this topic, but I have to stop somewhere!
There are various sources. D.W. Meinig's The Shaping of America, a Geographical Perspective on 500 Years of History is particularly good for the geographic history of America. The colonial era is covered in Volume I: Atlantic America, 1492-1800. Another book is American Boundaries, by Bill Hubbard, Jr. This book focuses tightly on how and why the boundaries ended up the way they did. It is kind of like the more famous How the States Got their Shapes, but more detailed, accurate, and scholarly.
(edit: thanks for the gold, woo!)
