Mark Norman in Devon is the go-to guy when it comes to all things black dog. He also has a podcast, and not surprisingly, he devoted an episode to the subject; it may help. Norman has also published a book with the imaginative title of "Black Dog Folklore", which came out in 2015; we can regard this as the definitive source on the subject. It's been awhile since I read the book and listened to the Podcast. Reviewing his book quickly for this question, I note a certain amount of "downsizing" of the dog. As the wilds cease to be as wild and so large, it is more difficult to place a large dangerous beast next to a community. It seems that Norman is arguing more for a disappearance as well as a conversion for commercial purposes of the Black Dog. More can be said on this, I'm sure, but this will give you a beginning.

edit: on a general note, it is useful to point out that beginning in the nineteenth century, other supernatural entities became less threatening. That certainly happened with the fairies/elves who shifted from the dangerous and frightening to the exceedingly cute: few if any true believers were concerned for the girls who convinced many that they had photographed fairies in the famous Cottingley incident, which began in 1917. A century before, there would have been widespread fear that the girls were in terrible danger. Finding a parallel "downgrade" of the Black Dog would be in keeping with this trend.