There are a lot of "first" cities that have been forgotten in time, but let's look at the ones you listed.

Jamestown, 1607 - We teach and talk about this one because it was the first Anglo settlement to remain permanently. It wasn't really financially successful, it was abandoned at one point, and everyone almost died. But they didn't and it stayed populated until established. Even less known is the sister colony to Jamestown also started by the Virginia Company (of Plymouth, the other branch from the Va Co of London that started Jamestown) that failed. Because, of course, it failed. Somewhat popular is the first failed civilian Anglo colony on Roanoke Island, but less known is the previous colony also set up by Sir Walter Raleigh on Roanoke and abandoned a few years earlier. It wasn't permanent so it doesnt get any sunshine in history class. The only reason we care about the 2nd Roanoke is because it failed as the first attempted permanent civilian colony (and the side story/legends behind America's first Anglo child, Virginia Dare and Croatan). We care about Jamestown because it was the first that worked.

Hampton, 1610 - Simply put, what makes it special? A strategic point and great land to develop, but it wasn't first. It didn't have the same struggle story as Jamestown. In fact Christopher Newport and the Jamestown originals identified that spot and came back in 1609 to build a fort. So it was even started by the same people, but it wasn't their first.

Newport News, 1613 - While Jamestown was struggling in June 1609, Christopher Newport was racing across the Atlantic with supplies. When a 3-day storm hit the fleet the boats were scattered and Newport's crashed in Bermuda. By the time Newport dealt with his subsequent shipwreck (which also had John Rolfe, the guy who started non-native tobacco growth in Virginia a few years later) and made it to Jamestown the situation was dire. It had taken about 10 months to get there and the winter of 1609-1610 is known as the dying time and for self evident reasons - the colony nearly failed. When he arrived the palisades were gone, the buildings had been burned for firewood, and a fraction of the settlers were left. They abandoned the colony and sailed down river until they met another man that was on Newport's ship, the new Governor Thomas Gates. He ordered them to turn around and return, which they did. The spot Newport and the settlers met Gates and recieved the news about returning with Gates and his supplies became, as you've now guessed, Newport News. Cool story, but it still wasn't first and was again the same people.

Albany, 1614 - Here we have the Dutch, who hired a guy named Henry Hudson that has some stuff named for him to do some exploring for a trade route, starting their first colony in America. The didn't hold on to it very long, losing it to the British in the 1660s. At that point the history of Albany became British history. It wasn't the first colony and was British - so what makes it important? If you're not Dutch, not much. It's like Fort Christina that was founded in 1638 by the Swedes. It was soon overtaken by forces from New Amsterdam, then taken by the English and even given to William Penn as part of Pennsylvania. Today we call an area very close to Ft Christina Wilmington, Delaware. But it wasn't the first Anglo colony in a land of Anglo history, so nobody knows about the fishing settlement.

Plymouth, 1620 - Now taking everything we've discussed so far, this one is obviously out of place entirely. It was started by Anglos "in exile" for their religion. It wasn't first. It didnt make a lot of money. It was ultimately absorbed into another colony. So why the Plymouth celebration? Humans and our need for symbolism. Plymouth faced a "dying time" their first winter (they started the colony construction on Christmas Day, 1620) as everything went wrong for them. Arriving off course they had no charter or structure where they were about to settle, so they wrote the first Anglo charter in the new world, the Mayflower Compact in November 1620 before scouting for a location. About half the settlers died by March. Then two weeks into March a Native trapper walked in and asked, in broken English, for a beer. Days later a treaty was signed and assitance offerd; that fall a harvest celebration would occur. In America today it is known as thanksgiving and was even recognized by George Washington as president. The colony remained and in 1741 a man that grew up around original colonists identified a rock as being significant - the first one the settlers stepped on according to what they personally told him, so he claimed. The Legend of Plymouth Rock was born and has captivated audiences ever since. Adding to this is the powerhouse that was Massachusetts Bay Colony, which absorbed the whole area in the late 1600s. The history then became Massachusetts Bay history, and the failed Dorchester Company of 1623 that settled at Cape Ann was absorbed into the Mass Bay Company when it formed in 1628, moving to Salem. Boston would come two years later. So Plymouth was the origin of Massachusetts which heavily influenced all things New England in colonial America much like Virginia did further south.

E for grammer