Other than novelty fireworks, the only "rockets' red glare" at the time of the American Revolution would have been halfway around the world. The British encountered rockets in Mysore, India in 1780, and were periodically shelled with them for all four of the Mysore Wars until 1799, when the Sultan, Tipu, was defeated. Tipu was apparently something of a technology buff and invested heavily in rockets. They were not particularly accurate. But they were very large and powerful, compared to the festive fireworks found in Europe. A barrage of them could be very frightening, and they could be quite destructive when they hit a good target, as in in 1799, when one managed to hit a British ammunition dump.

Some were brought back to England made a very big impression on Sir William Congreve, who set about making experiments of rockets based on them, pretty much at his own expense. He was able to argue their advantages: unlike a cannon, a rocket was quite light, and it had no recoil. For naval use, therefore, rockets seemed to be ideal. Why load a ship with thousands of pounds of heavy cannon, and then have it battered by their recoil, when it could carry the same weight of rockets and be more effective? While only a very large ship could carry enough cannon to be a threat to another ship of the line, even a small sloop could carry enough rockets to threaten a large frigate. Congreve had his chance, the government invested in them, and his incendiary rockets were employed during the Napoleonic Wars. Sometimes, if they had a very big target they could have great effect, as in the second siege of Boulogne ( 1806) and the naval attack on Copenhagen (1807), where they started huge fires. When the War of 1812 broke out, rockets in the British fleet also were found useful against the Americans. Ft. McHenry didn't fall to either rockets or bombs ( which , if they were only bursting in air, weren't a problem). But the shock and awe of a barrage of rockets at the Battle of Bladensburg in 1814 frightened and turned the flank of the American line. That led to the general rout of American forces and the British gaining complete access to (and freedom to burn) Washington D.C.

Rockets continued to be used by the British for the next few decades, and were also tried by other European countries. But they never became really accurate. Likely in part because it was difficult to control the rate of burn of the black powder propellant, make it uniform from rocket to rocket, to make their acceleration and velocity consistent. Until Robert Goddard began working on the problem, rockets also had problems with stability- Congreve would equip them with long tails , but that was not really enough ( though he seems to have wanted to experiment with making them spin, instead). But Congreve pretty much abandoned his rocket work after 1810, and died in 1825. His style of rockets were used....but after 1850, more and more wooden warships became armored and immune to rockets. As cannon improved in accuracy and power, Congreve-style rockets became obsolete.

​

William Congreve: A Concise Account of the Origin and Progress of the Rocket System (1810)

Roddam Narasimha: Rockets in Mysore and Britain, 1750-1850