The short answer is basically everyone hated Servetus's Unitarianism. In many ways he went further than even the most extreme Reformers and ended up being rejected by both Catholics and Protestants for the most part. He went very strongly against the Trinity, calling the belief an "error" explicitly (that's even the name of one of his books, De Trinitatis Erroribus). It's important to remember that even today almost all Christian groups accept the Trinity as essential, and it's as far as I know a concept that hadn't really been questioned by Christians in more than a millennium in Servetus's time. Shortly after Servetus (and possibly inspired by him), you do find some Unitarian groups--most prominently the Polish Brethren and the Transylvania Unitarians, and later the Unitarian Church in England from which contemporary Unitarian-Universalists are partially descended. But really, he had very few contemporaries (the other well known was Lelio Sozzini, known widely as Socinus--see also Socinianism, which is a traditional, somewhat pejorative term for non-Tritarian Christianity) and was preaching something that was essentially universally condemned as heretical by pretty much the entire spectrum of Christian theologians.

This small Unitarian movement, with whom Servetus is grouped though he was more of a thinker than a movement man, is classed by most historians today as part of the Radical Reformation, rather than the "Magisterial Reformation" of Luther, Calvin, et al. If I'm remembering George Hurston's argument correctly, there were three main wings of what he called the Radical Reformation--the Anabaptists (including modern-day Mennonites and Amish as well as more violent forms, like the Kingdom of Muenster), the Hutterites (who practiced a "Community of Goods"), and the various Unitarian movements listed above, as well as a few other small movements scattered here or there. But generally, the defining characteristic of the Radical Reform is that they were seen as completely beyond the pale of acceptable Christian thinking by both Catholics and most Protestants, and therefore suffered prosecution by both. I don't know specifically who else condemned Servetus, but if asked, almost every important theologian of his period would have. Even the English Act of Toleration of 1689 (a century and a half later), which legalized all "non-conformist" Protestants explicitly did not allow for non-Tritarian groups (or atheist or obviously Catholics). I believe the restrictions on Non-Trinitarianism in England (generally one of the more tolerant countries in Western Europe towards strange Protestant groups) were only lifted in 1813. I don't know who felt "threatened" by him, but most of his peers would have considered his ideas heretical and deeply dangerous. His ideas were only really allowed in the most tolerant places, like Poland (before 1658) and the Netherlands (though I'm not sure precisely when the Netherlands became the tolerant place for "dangerous" Protestant ideas, it long had that reputation).