Certainly Kelvin never said that — he authored a famous speech in 1900 on the "dark clouds" that lay over large parts of physics, making it clear that there were still things to learn out there, essentially indicating (in retrospect) several problems that would be solved by relativity and quantum theory — but Albert Michelson did remark in 1894 that "it seems probable that most of the grand underlying principles have been firmly established and that further advances are to be sought chiefly in the rigorous application of these principles to all the phenomena which come under our notice," which is a similar sort of sentiment. Max Planck was told as a young student in 1875 that physics was more or less "finished." But it is unclear whether these were really that common sentiments, or just anecdotes dragged out later when — to our great amusement — huge discoveries took place that rocked the entire foundations of the discipline starting around 1895 (with the discovery of X-rays).

The notion that everyone had become smug and content with physics, as the historian Helge Kragh puts it, is somewhat mythical. It's a story that people in the next generation told in order to ridicule the previous one (somewhat unfairly), and one that is dragged out whenever someone dare suggest that science may have figured out a few things for sure. But it's not clear it was all that common a sentiment, and by 1900 it was very clear that there were a lot of interesting "mysteries" in physics still to be solved.

Chapter 1 of Helge Kragh's Quantum Generations discusses this, and also gives a more complete discussion of what physics at the turn of the century looked like — not nearly as staid and stodgy as these kinds of myths would let on.