Under the Soviet Union, a giant experiment (still limping along) was undertaken to domesticate foxes.

This project, which started with 130 foxes (which were already the product of generations of fur farming), has gone through over 40,000 foxes over 40 generations of intensive scientific breeding and ended up with 700 highly domesticated individuals in 1996, before they culled the herd due to funding problems.

The resultant foxes are extremely submissive and eager for human affection. They are house-trained and willingly walk on a leash. To prevent the possibility of taming contaminating the breeding results, the pups are kept separate from human contact for much of their first few weeks.

To compare this program to any hypothetical premodern breeding program is problematic in the extreme. What monarch would pay for such a massive experiment? What successor would continue funding his father's folly? Without the Mendelian concept of heredity, would breeders take precautions to prevent merely tamed (as opposed to genetically domesticated) foxes from breeding? To produce a mostly domesticated breed of foxes took decades of highly focused work which benefited from modern concepts of heredity, genetics, and scientific breeding. Any directed effort at breeding domestic foxes would be unlikely without those aids.

Domestication of any animal is a painstaking process which takes generations, requires significant investment, and always carries the risk of injury or disease. There are about 5400 mammal species and only a couple dozen have been successfully domesticated. In almost every case, the effort needed to domesticate a species far outweighs any benefit, especially when selective breeding of an already domesticated species can alleviate whatever problem it is you're trying to solve.

Domestication, particularly of cats and dogs, is not thought to be an intentional project. In the case of dogs it is speculated that wolves followed groups of human hunters around living on scraps. The wolves that more easily integrated into the hunting group thrived, eventually evolving in to the gregarious creature many of us keep as pets today. Cats are believed to have become domesticated shortly after permanent settlement and agrarian societies formed as they lived off the vermin which thrived in human settlements.

Domestication occurred substantially earlier than the era immediately prior to industrialization (I only point this out because the way you phrased your question makes it seem like you think domestication occurred in the period immediately prior to European industrialization). Domestication of cats dates back at least 12,000 years and the domestication of dogs probably earlier. There are many varieties of dogs that are adept rodent hunters, such as smaller terriers. As dogs were already well along the path to domestication, why would one be bothered with taking an animal that is not at all domestic, like the fox, and domesticating it when an easier option is already available?

One of my favorite books on the history of dogs is Medieval Dogs, you might want to check it out to learn more about the role of dogs in pre-industrial Europe.

So you're asking why an animal that naturally hunts 'vermin' wasn't domesticated to make it hunt 'vermin'... do you see the flaw in your logic here?

Foxes are really good at being foxes, they are efficient hunters, scavengers and breeders - that's why the red fox is the most widely distributed mammal on the planet - they don't need any special human run training program to express their natural behaviour. I don't know how familiar you are with the red fox, but they are common where I live and your question is a bit like asking 'why didn't we teach sheep to eat grass so they can be lawnmowers?' It's just surreal.

Also, before modern veterinary medicine there were far more puppies and kittens born than could be put to use or housed, when people were killing unwanted litters of puppies and kittens why would they decide to bring in a new species for pest control? There was no gap in the market. Even in ancient Egypt where cats were worshipped there was enough of a surplus for ordinary people to be able to buy cats at the temple to be sacrificial offerings.