When did people stop believing in witches? What was the cause of this?

Speaking broadly, persecutions for witchcraft correlated with instances of confessional conflict, rather than just popping up sporadically because one neighbour accused another of cursing their cow. The notion of witchcraft in early-modern Europe was not, as we might imagine it now, one steeped in paganism, but rather one which existed within the intellectual framework of Christianity predicated upon the idea of Satan as the counterfoil to God (I'm not a theologian but that's the basic idea). Belief in witchcraft wasn't the preserve of the 'simple peasantry', but something discussed extensively by learned people - perhaps most famously, King James VI of Scotland (later King of England) wrote a treatise on the subject, Daemonologie, in 1597.

In the context of the Reformation and Counter-Reformation, Europe in the 16th and early 17th century is undergoing a massive religious upheaval, as territories and principalities cling to or break with the Catholic church, sometimes switching back and forth via conquest or insurrection. So this is an environment in which religious beliefs and practices are tightly watched and controlled; and the threat of Protestantism means that the Catholic church is on the defensive. It's not a coincidence that the areas of Europe which saw the most prolonged and wide-ranging witch trials were Catholic regions at imminent risk of Protestant conquest or uprising, primarily in western and southern Germany, e.g. Wurzburg and Bamberg, as well as the southern Netherlands and north-eastern France. In both of those cities, the trials only stopped when Swedish troops arrived under the Protestant king Gustavus Adolphus of Sweden. Securely Catholic countries such as Italy and Spain had few spectacular witch-trials (although they did prosecute heresy via the Inquisition). There were Protestant 'witches', but again, accusations of witchcraft flourished in areas and conditions where Catholic beliefs had never truly died out among the common people, despite the official embrace of Protestantism. So in England and Scotland, for example, the largest witch-trials took place in areas with high levels of Catholic recusancy: Lancashire (the Pendle witches), and Orkney - or against the backdrop of political instability, e.g. the persecutions of Matthew Hopkins, the 'Witch-Finder General' during the English Civil War.

So one significant part of the answer to your question is that the religious and political conditions in which witchcraft was seen as an expression of dissent or upheaval gradually resolved themselves, and religious conflict became a less pressing issue by the early 18th century. Another, more contentious theory is that 'enlightenment ideas' gradually led people to the realisation that witchcraft wasn't real. This is the theory which was espoused by Hugh Trevor-Roper in 'The European Witch-Craze in the 16th and 17th Centuries': that gradually the notion of witchcraft became ridiculous as scientific advances and rational thought allowed people to explain phenomena such as crop failures and mystery illnesses. This somewhat ignores the recorded persistence of belief in witchcraft in rural areas, well into the early 19th century, but it may explain why the educated authorities were less interested in prosecution - in England, for example, the Witchcraft Act of 1735 made it a criminal offence to claim that any person had magical powers or supernatural capabilities.