I do not know the scope of white fears around slave revolts, but I believe there was a substantial amount of fear to drive some areas of policy-making in the antebellum South. However, slave revolts could never be well enough armed, well enough coordinated, and with well enough numbers to make any slave revolt in the US successful. Where did the fear come from?
Two factors to keep in mind here. The first is that the fact there was no success so far is no indicator that success couldn't happen in the future. And even a failed uprising could be fairly extensive. This older response of mine I would point to for discussion of one such example, the German Coast Uprising in Louisiana.
More importantly though, the fact there wasn't a successful uprising in the US isn't the critical factor. The big driving force was the very successful uprising a little ways south, in the French colony of Saint-Domingue, now Haiti, an event which, among other things, saw over 20,000 white colonists killed during. When political leaders in the South invoked the fear of servile insurrection, it was Haiti to which they were pointing, either implicitly or explicitly. For more on this, I would point to this older answer of mine which looks at awareness and impact of the Haitian Revolution in the United States.