This is not a ridiculous question.

It is the central theme of Mary Kilbourne Matossian's book Poisons of the Past: Molds, Epidemics, and History published in 1989 by Yale - although psychedelic properties are only a fraction of the symptoms Matossian attributes to ergot (ergot is a mold which forms on rye). In her book she attempts to account for, well, frankly every major shift in European (and some American) social and economic activity from the late middle ages to 1800 including witch persecution. And for that reason she suffered an absolute demolition by social and agricultural historians as well as by biologists. This isn't to say that she doesn't raise interest corollaries of historical data which are suggestive - the problem is that the data is treated superficially, without historical context (for example she is dismissive of social-political analysis in the chapter Witch Persecution in Early Modern Europe), and without data depth or comparison, all of which lead scientists to question her claims to 'scientific methodology' and historians to question whether she actually knew much about history at all. In her own words from the introduction,

"I have made judgements on the basis of a little information about a lot of people."

Now, lest this seem entirely dismissive, I'll raise a point specific to your question. Matossian attempts to correlate the outbreaks of witch trials with the rye growing areas of (see her map p.72) period, and imputes based on generalities of climate that there must have been conditions for the growth of ergot mycotoxin and therefore this explains why witch persecution arose where it did. The argument is not buttressed whatsoever with details from witchcraft trials which could support the pathology of ergot poisoning. In fact, Matossian tries to build a case for ergot's relationship to witch persecution in 17th c Essex, England and ends up in a mess of logic. After stating that there are 'traces' of 'physical constrictures', 'fits' and 'gangrene' in court records which match the symptoms of ergot poisoning, Matossian then undercuts herself by suggestion everything is symptomatic of ergot:

"Most of the other reported symptoms could have been symptoms of ergostism, but they neither proved nor disproved the case in themselves [...] the symptoms included lameness; loss of sight, hearing, and speech; nausea; fainting; making animal noises, hallucinations; suicide, and sudden death."

Here she refers selectively to two minor secondary works only, out of context, and not to the primary source materials. (Furthermore there is no bibliography and no reference at all to key works in with persecution literature!) Not finding conclusive evidence in the attribution of every symptom ever to ergot in her case study geographic area, Essex, nonetheless Matossian marches forward drawing from outside:

"We have details of three case studies of bewitchment in England, none of which, unfortunately, occurred in Essex."

Here then is the crux of truly poor historical methodology: building a theory and then finding evidence to fit it. And so it goes with the problem of attempting to impute explanation into the historical record through a material scientific explanation without the careful work ingof data.

Historians (including scientists) have given a lot of thought to ergot's presence in the records, and we do find some moments where it seems that it is indisputably present: most memorably in my field are the chronic outbreaks in early 11th century Aquitaine where the monk Adémar tells us that sufferers of ergot poisoning, of sacer ignis, had visions of heaven and hell. Sacer ignis has a clear epidemiology in certain records in continental Europe, and could provide explanatory power in some localized cases over the centuries, much as the plague can more broadly, however, for Matossian's central study of witchcraft in England, the case falls flat:

"Her attempts to establish the presence of extensive ergot infection in Britain, which alone among European states has no indigenous name for that infection, and where documented outbreaks of ergotism are almost nonexistent, make unconvincing reading. The evidence of Erasmus Darwin (1791) that ergot was a "disease affecting rye in France, and sometimes in moist seasons' is more impressive: he needed to explain ergot to his British readers. It was not an explanation that was needed elsewhere in Europe, where the abortifaciet effects of ergot, both human and veterinary, were widely known to the common people by the mid-eighteenth century.

"Similarly, Matossian boldly claims rye as the staple of the English poor, ignoring significant regional variations in the consumption of different cereals and pulses, and without any attempt to match regional diets with regional mortalities." [Review, Anne Hardy, The Journal of Interdisciplinary History, Vol. 21, No. 3 (Winter, 1991), pp. 509-513]

The issue remains one of speculation with very thin foundation. There have been some inconclusive papers published in the last twenty years covering witch persecution and ergot poisoning in Scotland, Norway, and colonial America. The Norwegian witch trials of the Sámi in Finnmark 'learning' witchcraft by consuming it is to me the most suggestive and convincing because the author connects suspect rye and the manifestation of symptoms through witchcraft trial testimony. But no others I am aware of present evidence of connection other than 'rye was eaten in areas where there were witch trials, and some people claim symptoms that can be read as ergot', which reflects the challenge that "correlation does not equal causation." Moreover, it is clear that in the history of early modern judicial torture that confessions of epic psychedelic proportions can be obtained without moldy bread explaining it away.