There weren't really any 'Protestant Inquisitions' as such, because inquisitions were by definition a Catholic institution, and I am not aware that there were any Protestant churches or states who directly emulated inquisitions and referred to them as such.

However, if you're asking if there was organised religious persecution perpetrated by Protestants in the early modern period, then the answer is absolutely yes.

Religious Persecution in Early Modern England

The example I am most familiar with is that of England, where state-sanctioned persecution of Catholics and other religious minorities by the Protestant state occurred to varying degrees from the 16th century to the 19th.

There were two motivations for this. Firstly, there existed the idea that it was an act of kindness to persecute religiously incorrect behaviour and belief, because failing to do so risked tarnishing the immortal soul of the incorrect person. This idea has been referred to as 'charitable hatred', and there is an excellent book by Alexandra Walsham with this title, which I would recommend if you're looking for detailed sources on this topic. This idea gradually fell out of favour as religious heterodoxy became an unavoidable fact of life in the late 17th and 18th centuries.

Arguably more important, in the case of England at least, was the idea that allowing Catholics and other 'radical' religious minorities to congregate and worship openly posed a political threat to the state. When Elizabeth I took the throne in 1558, fears were very high that a Catholic conspiracy or invasion would dethrone her. Several hundred Protestants had been burned by Mary I's regime shortly before this, so English Protestants, particularly prominent ones in Elizabeth's government and church, believed that they would surely be killed in the event of a Catholic takeover.

This meant that persecution of Catholics was seen as a necessity not only because of their incorrect religion, but because it was believed they were potentially subversive and owed allegiance to the Pope rather than to the Protestant state. After the explosion of dissenting Protestant sects which emerged during and after the Civil Wars in the mid-17th century, this logic was extended to some Protestant dissenters as well, although they were then mainly tolerated after the Revolution of 1688.

This wasn't mass-killing on the scale we would think about it in the 20th century however. I have seen estimates of around 600 Catholics executed in England in the early modern period, which is obviously a lot, but still a very small proportion of the Catholic population of England. Mostly, it was clergy, particularly Jesuits, who were executed, because they were viewed as treasonous ringleaders for possible plots. Mainly, ordinary Catholics were persecuted through things such as imprisonment, fines, and exclusion from certain public offices. The exclusion of Catholics from these positions continued until 1829, and, in the case of the royal family, until 2013. However, this is true of the Spanish Inquisition as well, for example, where only around 2-3% of victims were actually executed.

Witch Trials

You asked about witch trials as well, and there the answer is certainly that Protestants were responsible for a lot of witch trials and executions, but its rather difficult to discern an answer as to whether Protestants generally persecuted witches more zealously than Catholics.Its certainly true that the most predominantly Catholic areas, such as Italy and Spain, did not have that many witch trials, and that Germany, a centre of Protestantism, had the most.However, its hard to exactly quantify the cause here, as this difference could possibly be accounted for not by religion, but by stability. Germany was absolutely devastated by a series of religious wars in the early modern period, culminating in the catastrophic Thirty Years War, and it isn't surprising that paranoia and persecution was more common in these devastated areas than they were in relatively stable areas such as Catholic Spain.

Its been argued that witch hunts were not peculiar to either Catholicism or Protestantism, but arose in areas where the two were in direct conflict. In south-west Germany, a hotspot for witch trials, the data appears to show that more trials actually took place in Catholic-controlled areas, but then Catholicism is generally stronger in southern Germany than northern Germany, so again its hard to draw any direct conclusions from this.

Conclusion

So, who was 'worse' in terms of persecution in the early modern period, Catholics or Protestants? Well, that's not really the kind of question historians generally seek to answer, or one that it would really be possible to answer even if we wanted to, since there are so many different factors involved.

In fact, I would argue that in general modern audiences tend to actually overestimate the severity of inquisitions and other forms of religious persecution in early modern Europe perpetrated by both Protestants and Catholics. Yes, there were some incredibly brutal outbreaks of persecution in the early modern period, but I think in popular consciousness these episodes are thought of as analogous to the genocides and political mass-killings of the 20th century, which they weren't really. They were far smaller in scale and actually often involved much 'milder' forms of punishment and persecution than would be popularly assumed.

Instead, the really devastating aspect of early modern religious conflict and hatred was war, and the famines, disease outbreaks, and population displacements that resulted from them. The English Civil Wars and associated conflicts in Scotland and Ireland, which were absolutely religious conflicts, although mostly not directly Protestant v Catholic, resulted in the deaths of probably around 200k people. In contrast, the combined death toll of the Marian persecution of Protestants, the Elizabethan persecution of Catholics, and English witch hunts was probably not much higher than a thousand. The number of deaths resulting from the revolt of the Protestant Netherlands and Catholic Spain's attempts to retain them in the Eighty Years War was probably over 100k, compared with maybe 5k killed by the Spanish Inquisition. The number of people killed as witches in Germany was maybe in the tens of thousands, but even this is dwarfed by the death toll of the Thirty Years War, which may have been as high as 8 million. Persecution at its most severe, such as the German witch hunts, was often a by-product of war and instability as well. Obviously ascribing 'blame' or 'fault' for these wars primarily to either Protestants or Catholics is even more problematic and unachievable than with persecution, as these were complex conflicts with multifaceted causes and often with multiple distinct stages involving constantly changing participants.

TL,DR: Yes, there was absolutely religious persecution perpetrated by Protestants, but neither that nor, in fact, the Catholic inquisitions, really resulted in a huge number of killings in the way that modern popular consciousness often assumes. Instead, religious warfare was far more significant in devastating communities and causing mass-death in this period.

Sources

You were hoping for books, so I will attempt to provide a few good ones. I already mentioned Charitable Hatred by Alexandra Walsham, which is great on the topic of religious persecution in England. Another good book on this is Persecution and Toleration in Protestant England 1558-1689 by John Coffey. For witch hunts, The Witch-Hunt in Early Modern Europe by Brian Levack is a little bit old now, but is very well-regarded and was released in a revised edition in the past 10 years or so I think. For the Thirty Years War and the devastation and persecution associated with that, Europe's Tragedy: A New History of the Thirty Years War by Peter H Wilson could be worth looking at.