This seems like it would be a much more complicated question than it actually is. Most academics, after pointing out that the Victorians were not really particularly prude, as there was discussion of sex and awareness of female orgasm, etc, point out that the idea of prudishness and hypocrisy comes from the second generation of Victorian writers, specifically Samuel Butler, especially in his Way of All Flesh and Erewhon which more or less accused his own parents (and thereby the whole generation) of hypocrisy and double dealing.

This was picked up by Pearsall, et al, in his Worm in the Bud, where he approached the entire Victorian era as one of hypocrisy and prudishness.

If you want a deeper look at his, Michael Mason and his Making of Victorian Sexuality is basically a book length treatment of this idea.

My main effort in this study is to bring out the extent to which the disparaged sexual moralism of the Victorians years grows from aspects of late nineteenth century culture which are likely to strike us as among the most bold, progressive, and refreshing features of the period... It is the chief oddity of our modern sexual code in the West that is has no place for the notion of sexual excess.. It is possible to understand and see the point of nineteenth century moralism if we realize that for many people issues about sexual behavior presented themselves as they do to modern feminists, as issues to be judged according to a secular, rational, and progressive code rather than in the light of the deliberances of religious and moral authority... In its sexual practices 19th century England emerges as a society that had some kind of crisis of confidence temporarily over courtship, marriage, and procreative intercourse in its first two or three decades

To add on to /u/AnnalsPornographie's point, part of the image of the Victorians as prudish comes literally from their images, particularly those from - in a not unexpected parallel - the late nineteenth century. European/American clothing changed enormously over the course of Victoria's sixty-four-year reign, running from the rather flamboyant 1830s (see "The City and Country Beaux", by Francis William Edmonds ca. 1838 at the Clark Art Institute) to the vibrant 1850s ("After a Long Cruise", by John Carlin, 1857 at the Met), the often quite bright or loud fabrics of the 1860s (two dresses in the Met), the somewhat fanciful 1870s ("Camille Monet on a Garden Bench", by Claude Monet, 1873, at the Met), and so on. While there were dark and plain clothes made earlier in the century, both men's and women's clothing for day dress became significantly more sedate and concealing as the century drew to a close. With men, this mainly was reflected in the waistcoat: early Victorian waistcoats could be almost garish (["John Haigh", by Henry Moore ca. 1840, at the Scarborough Museums and Gallery])(https://artuk.org/discover/artworks/john-haigh-8956/), while by the end of the century the waistcoat (during the day) was typically part of a matching three-piece suit.

With women, it was much less subtle. The wide neckline of the 1830s and 1840s shrunk to encircle the neck closely by the late 1850s, and by the late 1870s a standing collar was common; in the 1880s and 1890s, these could be fairly tall and cut to wrap around the throat and fasten on the side, so as to show none of the skin of the neck or as little as possible. (Compare the short standing collar here with the one on this bodice.) Likewise, we go from belling, open sleeves in the 1850s and 1870s (with a break for loose coat-style sleeves in the 1860s) to tight, fitted sleeves in the 1880s and most of the 1890s (with a break for sleeves with a large puff at the top - still tight in the forearm, though). Fashionable qualities for fabric had previously been lightness and airiness; over the last quarter of the century - when photography truly boomed - there's more emphasis on strong, dense fabrics. As a result, when people of the twentieth century looked back on images from a generation or two ago, they generally saw studio portraits of women in day dress with every inch of skin apart from face and hands covered with dark cloth. The early twentieth century saw fashion turn back to appreciating light, airy fabrics in everyday dress as well as showing more of the arms, neck, and upper chest, which influenced how people living during the time "read" those portraits - "if they didn't show what I, a person of 1932, consider a normal amount of skin, it's because they were prudes who scrupulously avoided all thought of sex." Additionally, the supposed rigidity of the late Victorian corset was thought by this generation to have prevented any kind of unseemly movement, and the supposed weight of a full outfit (particularly one with a bustle) was likewise considered to hamper women from doing anything but sitting decorously on a couch.

And I don't think I can finish without mentioning the oft-cited story about covering up piano legs because the resemblance to human legs was seen as indecent. This originated in Captain Frederick Marryat's diary of his travels in America in the 1830s as a quaint and over-refined practice specific to a girls' school in New York City, and there's no proof that it occurred anywhere else - or, indeed, that Marryat's anecdote was an earnest bit of ethnography rather than a funny joke for his English audience.