Edit: 1492***
As you're probably aware, the Spanish and Portuguese colonised large portions of South and Central America in the time between Columbus 'discovering' America and the first English settlement in North America, which was the so-called 'lost colony' of Roanoke in 1585. Previously the Spanish and French tried and failed to colonise the eastern coast of what is now the US, their colonies mostly failed due to disease, starvation and hostile natives. The Treaty of Torsedillas divided the New World up between Portugal and Spain, and the east coast of North America fell within the Spanish claim, so the Portuguese didn't attempt to colonise it.
There are several reasons that the English took a relatively long time to begin their colonisation effort. The first of these is that colonies were very expensive to establish, maintain and defend. English relationships with Catholic countries in mainland Europe were very poor due to Henry VIII's establishment of the Anglican Church and Elizabeth I's eventual excommunication, so it would have been very difficult for England to defend any colonies from attack, especially since England at this point could hardly be considered a great power. The English contented themselves with raiding Spanish galleons instead.
With the accession to the throne of James I in 1603, Anglo-Spanish and French relations mellowed somewhat due to his less radical brand of protestantism and the marriage of his son Charles I to a french princess. The fact that James I was also King of Scotland meant that the threat of invasion from Scotland was also lifted. The establishment of colonies was still prohibitively expensive (an attempt by Scotland to establish a colony in South America is estimated to have cost as much as 1/3 of all the money in the country) so much of the early colonisation effort was undertaken by private companies, albeit with the approval of King James, who issued charters such as this one in 1620 and this one in 1609 which gave the companies the right to govern their colonies and official status as territory of England.
Further Reading:
Nicholas Canny and Philip Morgan, The Oxford handbook of the Atlantic world c.1450-c.1850
There's a lot going on here.
It's probably necessary to unpack what you mean (or could mean) by "America" because the distinctions actually change the scope of what you're asking.
There's an idea of "America" in the context of transatlantic migrations that pretty much just means "land that is across the ocean from Europe." When people say that Columbus "discovered America" or something along those lines, they mean that he landed in what we'd usually think of as the Caribbean. This is kind of the same thing as calling it "the New World."
There's another idea of America as North America, the geographical entity. When people talk about the Vikings "discovering America," they're talking about interactions that occurred in what we'd think of now as eastern Canada.
And there's obviously the more narrow definition of America: the United States of America, either as a governmental entity or as a geographic area defined by that governmental entity. I think the whole cultural interest in the story of "the Pilgrims" draws specifically from the narrative's supposed import in constituting a national mythology or some other cohesive expression of American identity, so this is, I assume, the way in which you're conceptualizing of "America" in this question.
So, all the stuff happening at Plymouth Rock wasn't really the first instance of British colonists showing up in anywhere that we'd now call America. In 1584, Queen Elizabeth I gave Walter Ralegh (later: Raleigh) a charter to colonize "remote, heathen and barbarous lands, countries, and territories, not actually possessed of any Christian Prince, nor inhabited by Christian People," which he used to try to start up a colony in present-day Virginia North Carolina. The result of this was what people usually call the "Lost Colony" at Roanoke Island. It's not until 1607 that the British are able to establish a permanent settlement--Jamestown, Virginia--in what we'd now call America.
(I should also note that the British set up a variety of temporary settlements like fishing camps and seasonal harbors in places like St. John's so if you expand your definition of "America" a bit more, the history of British contact goes back to Cabot's "discovery" of Newfoundland in 1497.)
The Plymouth Colony wasn't the first British colony in the northeast, either--the Popham Colony, which was founded shortly after Jamestown, was located in present-day Maine, although that settlement was eventually abandoned. When the Mayflower landed in 1620, the area had been charted (but not settled) by Samuel de Champlain and John Smith. The Plymouth Colony was not, in its conception, a novel project, but some combination of the discrete narrative components that constitute our popular retelling--flight from religious persecution, endurance and perseverance through hardship, the symbolic taming of the heathen wilderness--resonates with people today who want to commemorate something representative of "American" cultural tradition.
It's true, in a general sense, that the Spanish (along with the Portuguese) were the most dominant European actors in the colonization of "the Americas," but this was primarily a difference of resource allocation. Britain, in particular, did stake claims to many parts of North America, but took considerably longer to successfully establish permanent colonies on that land. It's not until Charles I, who was considerably less tolerant of Puritanism than his predecessor, James I, becomes king in 1625 that there's enough large-scale migration to "New England" to actually establish a functional, permanent colonial structure--not just a loose network of trade-driven coastal settlements like Jamestown. This is all in contrast to Spanish colonization, which was fundamentally animated, in many cases, by mercantile as well as political conquest--to the extent that Spanish conquistadores were able to "claim" a large amount of territory for the Spanish crown, they were generally able to do so only through critical alliances (both economic and political) with indigenous groups. These are fundamentally distinct forms of colonization motivated, I think, by fundamentally distinct philosophies of how to best extract resources from the "New World."
So, tl;dr, the British were actually exploring the Americas around the same time as Columbus, British settlement in North America didn't begin with the Plymouth Colony, and British colonization was fundamentally distinct from Spanish colonization in approach as well as in execution.
If you're curious about Roanoke, I recommend Karen Kupperman's Roanoke: The Abandoned Colony for a comprehensive overview of the colony's own history, and The Roanoke Voyages, 1584-1590, ed. David B. Quinn, for a good amount of primary source documents.
For more about Jamestown, I recommend Frank Grizzard's Jamestown: a political, social, and cultural history
For more about the Plymouth Colony, I would suggest looking at Nick Bunker's Making haste from Babylon and, for a discussion of the Plymouth Colony's significance to American cultural tradition, see Godfrey Hodgson's A great & godly adventure
For more about the broader scope of British colonization of North America, see David Childs's Invading America, which gives some greater clarity to the notion of the "long sixteenth century," i.e. the time from Cabot to John Winthrop, as a period of continued British interest in North America. Childs also talks a good deal about why British colonies failed--he would probably argue that it wasn't a fundamental difference of philosophy that made British colonialism distinct from the kind practiced by Spain or Portugal but, rather, that the British were just profoundly unsuccessful at conquest.
That said, for more on the British and Spanish colonization strategies in America, see J.H. Elliot, Empires of the Atlantic World, and Lange et al, "Colonialism and Development: A Comparative Analysis" in American Journal of Sociology 1111.5 (2006), specifically the section "Theory of Colonialism" (1416-1423) that characterizes Spanish colonization as "mercantile" and British colonization as "liberal," each with specific, theory-attendant meanings.