In the United States much of the early, 1820s-40s and later ca. 1915-20s anti-Catholic sentiment stemmed more from the mass Irish and German immigration of the early 19th century than the actions of the Pope himself. Many Protestants viewed the Papal States combination of Church and State to be a form of heresy and to Protestants, Biblically speaking, It was the "Whore of Babylon" of Revelation.
Revelation 17:9 This calls for a mind that has wisdom: the seven heads are seven mountains on which the woman is seated.
Protestant ministers identified these "Seven mountains" as the seven hills of Rome
Below, a pamphlet published in 1835 (nearly the height of anti-catholic sentiment) shows some of Protestant fears, the author, Lyman Beecher, uses fear of a Catholic papal state in America to fearmonger against the new Catholic immigrants. They believed that this wave of immigrants, highly religious and highly political, would be swayed by the Priesthood to turn America into a Catholic nation.
But if, upon examination, it should appear that three-fourths of the foreign emigrants whose accumulating tide is rolling in upon us, are, through the medium of their religion and priesthood, as entirely accessible to the control of the potentates of Europe as if they were an army of soldiers, enlisted and officered, and spreading over the land' then, indeed, should we have just occasion to apprehend danger to our liberties. It would be the union of church and state in the midst of us.
One last reason is the apprehension by the Papacy to condemn slavery, which upset many abolitionist northern protestants, he would finally do so in 1839 in which Pope Gregory XVI issued the Papal Bull In Supremo Apostolatus
Edit: Sources: Religion in American Life: A Short History, 2ed. Butler, Wacker, Balmer, O'Toole, James M. The Faithful: A History of Catholics in America