If you've looked through the FAQ then you might have come across this post in a multipart series by u/Anthropology_nerd, and I can't recommend it enough as a jumping off point.

As that post title indicates, the issue is not so much that diseases hit indigenous people hard - they absolutely did (and still do, even with this year's pandemic). The myth is that massive indigenous depopulation was caused by disease alone. The most prominent version of this theory is the "virgin soil" theory, namely that indigenous peoples of the Americas, because of their isolation from the rest of the world, lacked immunities to Old World diseases, and when exposed to them died in extremely high numbers (the Jared Diamond version, as I recall, is that a relative lack of domesticated animals meant lower exposure to diseases). The rough implications of this theory is that mere contact with Europeans inevitably meant exposure to these diseases, which then raced far ahead of any zones of contact and rapidly killed upwards of 90% of the population on the continents. You can even find scientific models indicating that such a "Great Dying" caused the Little Ice Age, but these models rest on this rather dubious historic theory without providing any hard demographic evidence for it.

To get back to the historic consensus, diseases are absolutely recognized as a major source of indigenous mortality, but in combination with other factors that were part of direct European contact and colonization: warfare, dislocation, malnutrition and slavery, among others. These other factors weakened individuals' health and broke societies that otherwise could have recovered over time.

New England is a case in point. There is no doubt that a devastating pandemic swept some coastal communities around 1616-1618 (although it seems to have not affected other neighboring communities like the Narragansett), and it seems to have had origins among European sailors. However, these communities had had contact with Europeans for almost a century at that point, who among other things not only traded with these peoples but also took slaves (Tisquantum being a famous example). The estimated population natives in New England was about 100,000 in 1600 and this fell to the low thousands by 1700. In comparison the colonial population of New England hit 100,000 around that same time. Disease played a major role in that decline, but it was able to do so because of crowding native peoples off of their agricultural lands, crowding communities as refugees into defensive settlements during wars (which caused famine, malnutrition and allowed diseases to spread easily), difficulties "praying Indians" had in adapting to a new culture and social system while crowded into missionary towns (again allowing disease to spread), outright violence from warfare (about 20% of the Narragansett population - men, women and children - were murdered ina single encounter during King Phillip's War), and the persistent effects of slavery (large numbers of Wampanoag were deported to Bermuda and the Caribbean as slaves after King Philip's War).

So clearly disease caused a major role in native deaths. But taking it as a cause alone obscures more than it illuminates. To provide an example from my area of expertise, it would be like talking about the typhus epidemic in Russia of 1918-1922 which killed millions, but ignoring the famine and civil war that occurred during those years which weakened people's immune systems and society's ability to respond, and which therefore allowed the disease to claim so many victims.

u/Kochevnik81 did an amazing job answering this question, so I only have a few further points. I haven’t read This Land is Their Land yet, so I can’t speak specifically to the claims of the book, but I can speak more generally about the status of the impact of infectious disease debate.

As u/Kochenvnik81 mentioned, current scholarship does not deny that epidemic diseases hit, and continue to hit, indigenous communities disproportionately hard. The current best evidence suggests disease never acted alone; pathogens were part of the mix of toxic colonial insults, combined with warfare, slavery, territorial displacement, environmental transformation, and the breakdown of traditional social structures that decreased indigenous populations size and prevented demographic recovery. The current pandemic dramatically underscores how the hidden structural inequalities of a society can cause disproportionate harm to some, while leaving others relatively unscathed. Now, as in the past, structural violence increases host frailty before the arrival of a pathogen. What we see on the New England coast in the early seventeenth century is merely the first manifestation of the violence of settler colonialism that would gradually unfold across the continent.

Prior to roughly the early 1980s scholars oscillated between “low counters” and “high counters”, the low counters believing the Americas were lightly populated before contact, and the high counters suggesting the opposite. The high counters reconciled the disconnect between belief in a large pre-contact population size and the small number of Native Americans left alive by 1900 by emphasizing disease as the main cause of depopulation. The idea worked its way into the mainstream, most famously with the publication by the ornithologist you mentioned. Death by disease alone as a narrative was a convenient way to ignore the very obvious harm of colonial endeavors, overlook the pervasive violence and erasure of Indians throughout the colonial period, and ignore hundreds of years of of persistence, resistance, rebellion, negotiation, and vitality of indigenous communities.

Recent scholarship rebels against this erasure. Put bluntly, the best evidence from ethnohistory, archaeology, bioarcheology, and anthropology suggests “a generation of scholars has significantly overemphasized disease as the cause of depopulation, downplaying the active role of Europeans in inciting wars, destroying livelihoods, and erasing identities” (Beyond Germs, p.3).

Unfortunately, we are all novices outside of our areas of expertise, and there are moments in time when epidemic disease did strike Native American populations with disastrous consequences. Scholars not directly immersed in the demography/disease debate, even ones I highly recommend and whose work I love, are just starting to wrestle with this mindshift, and will default back into a “disease alone” narrative. I don’t know if Silverman falls into this category, again I haven’t read the book, but the narrative is so pervasive, and hard to ignore, especially when written evidence of epidemic disease abounds.

The populations of coastal Massachusetts, the Wampanoag specifically, were desperately trying to maintain their lands and position in southern New England during the growth of the Massachusetts Bay Colony. They were subject to frequent outbreaks due to proximity to colonial outposts that were unique, not the norm, for indigenous populations in the early seventeenth century. Only coastal Virginia near Jamestown, groups along the St. Lawrence River, and mission populations in Spanish Florida would have faced such an early impact. There is no evidence the first epidemic (the one that prompted Massasoit to try to ally with Plymouth) spread any further inland from the coast (Beyond Germs, p.152). The first verifiable smallpox epidemic that spread deeper inland and struck the Connecticut River Valley hit in 1633. For the southeast it would take another sixty years for smallpox to find it’s way into the heart of the continent (Kelton).

The norm for most of the continent beyond the frontier, for roughly three hundred years, appears to be periodic waves of epidemic disease with varying ranges of geographic spread, sometimes with generations of statis between waves, and time to rebound. This rebound is critical. Humans are demographically capable of replacing catastrophic losses, as long as other sources of excess mortality are limited and access to resources is maintained. What the Wampanoag faced, and what indigenous populations across the continent eventually encountered as the frontier drew ever closer, was excessive mortality (from multiple causes), as well as a lack of time and resources needed to replace those losses. In many cases the destruction and restriction of resources was intentional, a tool of war to decrease resistance in the hopes that hunger and disease would follow.

To sum up, disease never acted alone, and as a discipline we are only just starting to bring this idea into the mainstream. I share your frustration with authors who cite high epidemic disease mortality, but don’t dive deeper into the causes of that mortality. If we never examine the active forces that weakened populations, and allowed for disease to spread, we will never arrive at a full reckoning of contact.

Sources:

Cameron, Kelton, and Swedlund, eds. Beyond Germs: Native Depopulation in North America

Paul Kelton Epidemics and Enslavement: Biological Catastrophe in the Native Southeast, 1492-1715