This is not exactly my field, but is rather tangential to it. Either way, I know exactly where you'd begin to get a really good answer: Michael Mason's The Making of Victorian Sexuality and Making of Victorian Sexual Attitudes, really remain the best texts on this topic.

In his research he argues that victorians were not necessarily prudes, despite being characterized that way by their children (the generation after them), and the later Victorian authors (such as Samuel Butler). Instead, he argues, that Victorians just weren't that much into discussion of sex and sexuality. It didn't interest them that much, and it didn't seem necessary to discuss it in public.

On page 7 of Making of Victorian Sexuality Mason argues that

In its sexual practices 19th century England emerges as a society that had some kind of crisis of confidence temporarily over courtship, marriage, and procreative intercourse in its first two or three decade. Thereafter, in the mid-century at least, intercourse within marriage or relationships of concubinage to some extent displaced traditional resort to prostitutes and perhaps other kinds of casual sex.

Great changes did happen however--for example, in 1786 it was fine for a duke to introduce his mistress to Queen Victoria, but by 1802, Charles James Fox had to marry his mistress before she could be introduced into polite society. Another example was the breaking up and elimination of Cock and Hen clubs, which were more or less the 18th and 19th century equivalents of nightclubs or bars, where young men (cocks) and women (hens) could meet, drink, and perhaps have a little 'fun' afterwards.

The period between 1800-1850 saw huge changes in the formalization of public morals, public behavior and enforcement of that behavior. Rules for public behavior began to be laid down by the 'burgeoning' middle class. This period also saw the creation and founding of many police departments and constables in many municipalities in england. Many of these police departments were founded by religious groups who sought to reform public behavior first and foremost, in order to create a polite society.

One person who lived through all of this, and wrote about it, was the really fascinating Francis Place, who discussed the reformation of public behaivor, and the closing of Cock and Hen clubs, the founding of the police. Mason, and many others, base their research off of his journals. Anyhow, Place argued that the Victorians didn't really change that much in their private lives, but public lives were completely upended, formalized and regulated. He explains several reasons for these changes:

1. better regulated police,

2. the employment of women in the cotton industry,

3. the rapid increase in wealth,

4. the French Revolution, which broke up many old ideas,

5. the desire for information on the government (newspapers, political conversation, etc)

6. the rise of reading and political clubs,

7. the rise of Sunday, national, and other public schools.