To quote myself from a past answer:

A big part of that has to do with fiction. The Gothic genre was invented in the late eighteenth century, originally as a return to medieval standards and medieval or early modern (and typically Continental) settings, and soon combined with the burgeoning Romantic movement to encompass the kind of wildness, horror, and excitement that was typically not a part of everyday life. The Mysteries of Udolpho (Ann Radcliffe, 1794), for instance, is about a poor noblewoman in late-sixteenth-century France who is orphaned, menaced by her Italian uncle, imprisoned in a castle, and escapes to marry her true love - there are bandits, long-lost heiresses, and ghosts. Radcliffe's earlier but less famous novel The Romance of the Forest (1791) is another Gothic novel, about a French family that goes into hiding in a ruined abbey to escape their creditors, encountering a skeleton in a chest, a girl thrown out of her home and blocked from her inheritance, a frightening account written by a seventeenth-century prisoner, attempted rape and murder, rescues, trials, and suicides. As people continued to write in the genre through the nineteenth century while largely using contemporary settings - frequently as lowbrow but widely consumed "penny dreadfuls", but aspects of it turn up in popular-and-now-classic novels like those by Dickens - we tend to associate highly dramatic fiction featuring improbable coincidences, supernatural events, and dark atmosphere with the period.

Twentieth-century views of the Victorian era picked up on the popularity of this type of fiction. And in general, "moderns" saw Victorians as a benighted people, mired in restriction and ignorance which made it ripe for portrayal in horror. There are a lot of aspects of the period that contributed to this, outside of fiction! The late adoption of germ theory, interest in irrational and unscientific Spiritualism and seances, perceived repression of sexual urges and unsuitable romantic love, clothing that seemed unnecessarily complex and uncomfortable, extensive standards for mourning clothes and behavior (standards that predated the period, but were and are still particularly associated with it), florid decorative arts movements making great use of dark colors, strong beliefs in women's place in the home rather than the workforce ... If you wanted to depict something that a scientifically-minded person of the twentieth century would simply not believe in their own sterile, white and chrome kitchen, it could be much more compellingly done in a rambling and poorly-lit mansion decorated with dark wallpaper and intricately-carved wood paneling (particularly to a Victorian protagonist, but as in The Haunting of Hill House, modern characters could be put in these disconcerting settings). In 1948, a reviewer for the London Standard astutely noted:

Sustained by countless novels and plays and eagerly seized on by the makers of films is the Great Victorian Myth. The Myth has almost succeeded in converting the younger generation to a belief that Victorian domestic life was full of thwarted motherhood, tyrannical husbands and fathers, and spiritual frustration in dark, rambling houses. ... Clearly the thing to do was to torture the heroine, and if you put her in crinolines you could torture her much more severely.