Human's aren't rational under the best of circumstances. It is actually quite easy to force false confessions by applying various methods of psychological and bodily torture. This is something even today's justice system can easily cause by creating an atmosphere of hopelessness, isolation, fear and the promise of relief and reward through a confession. The methods used during with hunts were far less inhibited in comparison.

A good example is the case of Mathew Hopkins, self-styled Witchfinder General, who caused the execution of about 300 people in just fourteen months between 1644 and 1646 in eastern England (East Anglia). Confession under torture were inadmissible in court back then but Hopkins toed as close to the line as he could under the law (and lied if he couldn't) and that was more than enough.

The accused people were first kept without sleep or food for several days to see if a familiar would appear to help them.

This method of sleep deprivation was soon forbidden to him, however this quote indicates that it was still used, he just blamed the victims' inability to sleep on them being witches.

In the infancy of this discovery it was not only thought fitting, but enjoyned in Essex and Suffolke by the Magistrates, with this intention only, because they being kept awake would be more the active to cal their imps in open view the sooner to their helpe, which oftentimes have so happened; and never or seldome did any Witch ever complaine in the time of their keeping for want of rest, but after they had beat their heads together in the Goale; and after this use was not allowed of by the judges and other Magistrates, it was never since used, which is a yeare and a halfe since, neither were any kept from sleep by any order or direction since; but peradventure their own stubborne wills did not let them sleep, though tendered and offered to them.

They were then searched for a so called Devil's mark, an abnormal spot on their skin from which a familiar would suckle blood. Usually this was a mole, birthmark, a third nipple and the like. If no mark could be found the accused would be pricked with knifes or special needles to look for a piece of skin that was numb to pain since that was also a sign of a Devil's mark. Many people were also forced to walk or run without stopping for hours on end until their feet were blistered. Or they were tied to chairs in extremely uncomfortable positions

Most victims would confess to everything they were asked to make the ordeal stop. Add the hunger, sleep deprivation and complete mental and physical exhaustion and you had an excellent recipe for all kinds of confessions, no matter how outlandish and patently false.

Hopkins wrote a book about his methods and it was often used in later witch trials as well, like in Salem. If you are interested in this, I would recommend that you read his The Discovery of Witches in which he answers several questions and accusations about his methods and defends them. It's not overly long and fairly easy to read: http://www.gutenberg.org/files/14015/14015-h/14015-h.htm