I think your premise is flawed. There isn't any evidence that theater was considered immoral in the 19th Century, though I imagine there were naysayers of the period just as there are naysayers today who deride TV and video games and whatever else is popular.
Here is a book from 1852 called "Wemyss' Chronology of the American Stage, from 1752 to 1852". It gives a timeline of theaters opening all over the country during that period. You'll also notice that the first mention of an opera house is this entry on page 173:
"The twelfth theatre built in New-York was the First Italian Opera House, corner of Leonard and Church-Street...in 1839..."
Looking through the other entries in that book for "opera", it seems that opera houses were as highbrow then as they are today.
I can't find any evidence that "most towns" had opera houses instead of theaters. In fact, as early 19th Century city directories present it, theaters were ubiquitous throughout America while opera houses were virtually non-existent: