Do you mean a penal colony? For the most part, no, the British colonies in the new world would not be considered penal colonies. Penal colonies are typically characterized by a majority or high percentage convict population (compared to other peoples). I would also argue that penal colonies have state-run (or state contracted) penal sites, such as forced labor camps. There is an argument that can be made about the role of convicts on plantations or in iron forges as sites of coerced labor, but as these were private enterprises where convicts worked alongside slaves, servants, and free people, I would argue they are disqualified. A key goal of penal colonies was to isolate convicts from the population, whereas the sites of convict laborers in the British colonies tended to have mixed labor forces.
The British state did sentence and transport large numbers of convicted persons to the colonies, and sold them into forced servitude in the new world. However, the number of convicts did not exceed the number of people coming voluntarily, either as indentured servants or free passengers. In the seventeenth century, the year with the greatest number of convicts transported was 1685/1686, as a result of the Monmouth Rebellion. About 1,000 foiled rebels and convicts were transported to Jamaica and Barbados, but after arriving, these convicts (and ones shipped the previous years) made up less than 5% of the white population in Jamaica, and less than 4% of the white population of Barbados. The only North American population census we have that explicitly lists convicts is the Maryland census of 1755, where convicts made up less than 2% of the population.
Between 1607 and 1783, perhaps as many as 60,000 convicts were transported by Britain to her colonies across the Atlantic. About 50,000 of those were transported between 1717 and 1776. For the period before 1717, the majority of convicts were shipped to Jamaica and Barbados, with limited numbers arriving on the North American mainland. After 1720, the vast majority of felons were transported to the Chesapeake or Delaware valley (VA, MD, and PA). Based on estimates by Aaron Fogleman, about 300,000 Europeans emigrated to the thirteen North American British colonies between 1717 and 1775. That would mean that one out of six Europeans travelling to the British colonies was convicted of some crime, and sentenced to be transported to the new world. Fogleman’s total estimate is likely on the high side, with a more conservative estimate would be 250,000 Europeans. Either way, the maximum percentage of convicts out of total European emigrants would be 20%. And again, these were just numbers of newly arriving colonists. The mainland colonies had populations that far outweighed new arrivals in the eighteenth century. The British courts sentenced about 80-85% of convicts to 7 year terms of exile (typically accompanied by a concurrent term of labor), so at any one point, convicts never exceeded about 5% of the European population in any colony.
Sources:
Data drawn from Coldham, British Emigrants in Bondage (2005)
Fogleman, "Migrations to the Thirteen British North American Colonies, 1700-1775" Journal of Interdisciplinary History (1992)
Ekirch, Bound for America
McCusker, "Rum Trade and the Balance of Payments of the Thirteen Continental Colonies" (Ph.D) (for population of the colonies)
Quintanilla, "The Monmouth Rebels in the West Indies" (Ph.D.)
Anderson, ed. A Global History of Convicts and Penal Colonies
Tomlins, Freedom Bound