People used "chew sticks" prior to the invention of the toothbrush. The chew stick was basically a twig with a brush-like end. The twig was literally chewed to remove the bark and fray the fibers. This practice dates as far back as the Egyptians and Babylonians, and is still used in Africa and the Middle East.
The modern toothbrush was invented in China. The Chinese embedded hog hair bristles on handles made of wood or bone. These early toothbrushes were exported to the West around the 17th century, but were too stiff for European consumers, so horse hair replaced hog hair.
My source is Panati's Extraordinary Origins of Everyday Things (1989).