While people understood that glass ( and some jewels) could magnify images pretty early, optical glass making and lens making did not really develop until much later. The first practical telescope is Galileo's, in 1609, and the 17th c. is the time of great experimentation with optics. Christian Huygens designed telescopes, Leeuwenhoek was working with early microscopes, and the philosopher Spinoza was making his living making lenses. Achromatic lenses did not even come in until the 18th c.

As to why no one would put a telescope on a rifle until the 19th c., good question: but it would have been rather difficult. Telescopes of the 18th c. were not particularly rugged, focused with sliding the sections to make them longer or shorter, and scopes for rifles have to be rugged. Another possibility is simply the practical range. Muzzle-loading rifles would typically be used at ranges of no more than around 250 meters- the trajectory of the ball is pretty steep- so setting up a scope to accommodate a lot of change of elevation was likely pretty daunting. Even today, with flatter trajectories, scopes are adjusted for one range, or equipped with reticles that make it possible to compensate when aiming. A those ranges, with that steep a trajectory, an adjustable peep sight would have been much easier to adjust and use than a scope.