The abundance of chestnuts in eastern forests is well known, but the degree to which they (almost certainly must have) contributed to the diets of Native American peoples is actually not all that well understood. This is mostly because of preservation issues.

What we know about Native American diets has to be pieced together from oral history, ethnobotanical data, and archaeological evidence. The farther back you go, the more you're dealing with archaeological data, and that means a partial record.

Preservation issues are a huge factor in how much we know and what we know.

For example, we are pretty confident that Native Americans in eastern North America exploited whitetail deer and hickory nuts pretty heavily. Why is that? Because whitetail deer have reasonably large bones that preserve well in archaeological sites, and hickory nuts have dense shells that appear to have been regularly used as fuel in fires after they were smashed for their meats, and so burned but were carbonized rather than being turned to ash.

Similar to chestnuts, we are less sure about smaller animals, or insects (for example) which were abundant and a likely source of nutrition, but which don't preserve very well (insects almost not at all). So what this means is that we probably are missing a lot of information about what people were eating, simply because it never preserved. Were deer a big part of the diet? Well, we suspect so, because they're one of the larger prey species, post-Holocene, and would have been an attractive source of meat. But so were wild turkeys, raccoons, turtles, snakes, birds of all kinds, possums, etc. Not to mention fish. These animals are smaller, with more fragile bones, and so while we know they were eaten, we don't necessarily know in what proportions relative to deer.

With respect to nuts, and specifically the question... hickory nuts are one of a number of available mast (tree nuts / seeds) species available in the Eastern Woodlands. Others include pecans, walnuts of several types, acorns of a number of types, hazelnuts, chestnuts, and beech nuts. Hickory nutshell is almost ubiquitous in pre-Contact / pre-colonization archaeological sites, and we presume that they provided a significant proportion of calories. Hickory nuts contain abundant fats, and productive hickory stands could have provided nutrition sufficient for a number of people.

But the other masting species would not have been overlooked. Walnuts are also high in fats. Acorns are high in carbohydrates, and oaks-- the many varieties-- also are abundant in oak-hickory forests (obviously from the name). Both oaks and hickories have cyclical years of boom and bust production, but oaks of different types have different cycles, so when one type is not producing so well, another may be.

Ditto with hazelnuts, beech nuts, and chestnuts, which are all high in varying proportions of carbohydrates and fats. Chestnuts are high in carbohyrates, are really tasty, and can be used in a lot of different ways. They were probably a significant source of calories for millennia.

But the problem with acorns and other similar mast species is that while hickory and to a similar but somewhat lesser extent, walnut have relatively thick, durable shells that preserve reasonably well when burned / carbonized, acorns, hazelnuts, and chestnuts have thin shells relative to their size. Anyone who has ever roasted chestnuts knows that chestnuts have especially thin shells.

These are bad characteristics for preservation. The husks / shells can be reduced to ash more easily than hickory or walnut.

The potential issue there is obvious. We may be massively overestimating the importance of hickory over other less durable, less likely-to-be-preserved mast species. It's of such a concern for people who attempt to reconstruct dietary profiles from archaeological assemblages that some paleoethnobotanists have tried to develop predictive equations that use the presence / frequency of hickory nut shell relative to other species, and spit out actual proportions represented at a given site.

They suggest that proportions need to be adjusted considerably to reflect actual dietary composition. But it's frankly too difficult, and too uncertain, to estimate so many different species.

Turning to chestnut... chestnuts are enormous relative to their shell thickness. The shells are thin, and do not preserve well. It is guaranteed that we have under estimated, probably by quite a bit, the contribution that the resource made to the diet of indigenous Americans. I've long suspected that chestnuts were a huge part of Native American diets, but the data just don't provide a satisfying conclusion to that hypothesis, because preservation is not our friend.

There is certainly more that can be said on this topic, but you might consider starting with this answer by /u/louji while waiting on a response. The linked post offers a book that may be useful.

Just a brief clarification as an expert in host-microbe interactions specializing on plant disease (I have a PhD on the subject): your questions include "When the chestnut trees died, did people starve? How integral were these trees in the economy/diet? Was it a big deal when they vanished?", but I think this may misunderstand the timeline of Chestnut blight, as caused by the fungus Cryphonectria parasitica, in America. As described in a brief history of the disease from the American Phytopathological Society, C. parasitica was first detected in North America in 1904, and had not caused the widespread vanishing of the American Chestnut until the early 1940s. As such, the blight would not have affected "colonials." Having said that, I have enjoyed reading the discussion of the chestnut's impact on Native American diets.