Well, one answer to why psychology was more popular in the 1880s than previously, is that psychology's official origin date is often held to be 1879. This was the date when Wilhelm Wundt started the first experimental psychology laboratory focused on generating new research (with William James in America also having a parallel claim). Before this - with exceptions like Gustav Fechner's psychophysics or Wundt with his 'thoughtmeter' - psychology was essentially just a branch of philosophy (and one which had seen a lot of philosophical work since Descartes had identified consciousness as the founding principle of epistemology with the whole 'I think therefore I am' thing).

However, the kind of psychology that, say, Wilhelm Wundt did - focused on determining the nature of the mind, not necessarily on mental illness - is not really what you're asking about, I think. Instead, I think what you're largely interested in is psychiatry. For context, the usual definition of 'psychology' is 'the scientific study of mind and behaviour' - it's a scientific discipline which aims to understand how humans think and behave much more generally. In contrast, psychiatry is the subdiscipline of medicine, in which medical doctors specialise in mental health. There are also specialists within psychology, clinical psychologists, who focus on mental health, but clinical psychologists, more or less, date from after World War II.

Medical doctors who've been focused on scientific/entirely physical explanations for mental illness (something which distinguishes them from more magical approaches to mental illness, which have likely been around as long as mental illness) have been around since Hippocrates and people in his tradition, but it was around the turn of the 19th century that you get 'moral treatment', in which treatment of the mentally ill by medical doctors was focused on putting the mentally ill in more healthy situations and coming to understand who the patient was and where they were at - previously, to the extent that doctors paid attention to mental health, they had often advocated mental health treatments as advanced as bloodletting and chaining people up in mental asylums. This kind of moral treatment became rapidly more common over the course of the 19th century (leading to problems of its own, as the asylums of the era became very overcrowded and thus not particularly healthy as per their original purpose).

Even with the problems caused by overcrowding, you could certainly argue moral treatment led to an increased medicalisation of mental illness in Western societies where the moral treatment movement had been successful. Previous to this, mental illness was rarely seen as or treated as medical, but was instead treated within a community (rather than by doctors) and was seen as a combination of afflictions of the spirit and/or simply inherited and unchangeable (and so not worth medically treating). By the 1880s, you begin to get a particularly medicalised idea of mental illness, as advocated by the likes Emil Kraepelin, who framed diagnoses like dementia praecox (what we'd now call schizophrenia) as medical diagnoses in a way that would look familiar to those familiar with modern DSM diagnoses, and who attempted to find causal factors causing these mental illnesses (e.g., Kraepelin thought that schizophrenia was effectively early onset dementia).

Similarly, over the course of the 19th century, the understanding of the physiology of the brain and the nervous system increased in leaps and bounds. You get widely publicised cases like Phineas Gage, who survived a railway spike through the head, but whose personality was seemingly altered, or 'Tan', a French man who had brain damage to a part of the brain that's now called Broca's Area (after Paul Broca, the doctor who investigated Tan's brain after his death). Tan's brain lesions made him unable to say any word but 'Tan'; while this now reminds us of a certain Game of Thrones character, it showed in interesting ways that the brain was specialised to do different tasks to some extent. Similarly, a young Sigmund Freud in the 1870s did physiological research in the laboratory of Ernst Brucke, comparing aspects of the human nervous system under a microscope to equivalent bits of animal nervous systems (Brucke's work helped pave the way for the discovery of the neuron as the basic component of the brain was first figured out in the late 1880s by Santiago Ramon y Cajal).

Additionally, in 1859, you get Darwin's Origin Of Species, which implied that humans were simply evolved creatures that were entirely material in origin and nature - including our brains and minds. This was unsurprisingly a big influence on early psychology and psychiatry (and especially on Freud, whose theorising about sex is directly derived from Darwinian ideas about sexual selection).

Furthermore, during the 19th century, you get the increasing professionalisation of the university along the German model, which was very influential worldwide, and which provided a model by which there could be professional scientific and medical researchers. The medical profession, in the German model, was increasingly focused on scientific research and how that could be applied to the treatment of patients, in a way that was not previously emphasised so strenuously. During the 1870s/1880s, to again use Freud as an example, whilst a student at university he fairly seamlessly transititioned from physiological research to graduating as a medical doctor when he could not secure a full-time position in Brucke's lab.

This all came together, unsurprisingly, in a certain level of public curiosity about psychology and psychiatry, though popular interest in psychology and psychiatry has basically continued to increase across the 20th century and beyond, seeing we live in a world where people are 'triggered', perhaps by other people with 'inferiority complexes' who only think they have high 'IQs', and where we're 'reinforced' for acting in certain ways by governments full of 'narcissists' and 'psychopaths' who think they're 'idiots'. Language that originated in psychology and psychiatry regularly becomes part of the general discourse in a way it didn't quite in the 19th century. But, of course, there has always been a lot of interest in mental health, because there's always been a lot of people who have suffered with the symptoms of mental illness; the 19th century was simply when it started to seem like medical doctors might be able to use their increasing knowledge of the brain and the mind to actually help people.