First, in case someone is interested in reading the books I mention below, I've kept this post spoiler free. I do mention some broad things but avoid any plot details.

The phrase, "the butler did it," comes from what's typically called the Golden Age of Mystery or the Golden Age of Detective Fiction, 1920s–1940s. While the authors of this era themselves were not all English (Raymond Chandler and Dashiell Hammett were American), the English style of the murder mystery had a particular set of traits that differed from the American style. Writers like Agatha Christie, John Dickson Carr, G. K. Chesterson, and Dorothy Sayers, all prized the puzzle of the case over characters. Where Hammett and especially Chandler created unique characters and had a flair for turns of phrase, the English writers tended to be much more straightforward, crafting dense mysteries where it often appears no one could have done it (i.e., Murder on the Orient Express, the Hollow Man). The criminal has pulled off an elaborate "perfect crime," until the detective happens along. What's more, British detectives, such as Poirot or Father Brown, are rarely portrayed as in any danger, despite spending a lot of time in close quarters with blackmailers, murderers, and thieves. Chandler's Philip Marlowe, by contrast, is frequently beaten, drugged, shot at, and tangles with the seediest of the Los Angeles underbelly.

"The butler did it," reflects a few different stereotypes of the English mystery genre. First, there's often a wealthy family or victim, often in a manor house where a butler, a maid, and other house servants might live. This allows for isolation where characters might be trapped together in a snowstorm or somehow otherwise detained at a country house. (The trope is so common that parodies are abundant in film: "Murder by Death," "Haunted Honeymoon, and "Clue" are some obvious examples.) Because the mystery and the twists and turns are valued above all, the butler is a character who can be frequently present without necessarily drawing a lot of attention. While readers are focused on family members or other characters, the author can drop quiet hints about the butler through his actions in sentences that are seemingly unimportant. It's been famously said that to solve an Agatha Christie mystery, just focus on the person least likely to commit the murder, or the person who has the most airtight alibi. In the case of the butler, the former applies. Again, motive and the depth of the character were less important than the intricately crafted mystery the writers had created. Readers didn't want to come away invested in a character so much as they wanted to be surprised by the solution or, even better, pat themselves on the back for solving it.

In reality, the butler rarely did it; at least not in the more famous writers' mysteries. But the trope is useful in understanding the genre as one where lesser characters are guilty as a way of surprising readers. The writers whose works remain popular today "obeyed the rules," as it were. They usually didn't introduce last-minute characters or plot details, unless the clues were laid early on. For example, the butler won't turn out to be the long-lost brother of the victim, unless clues have been planted to let readers discover it on their own. Copycat writers would sometimes employ these tactics of last minute reveals, disappointing readers.

Finally, I'm being broad when I say the puzzle is the most important part of English detective fiction. In the case of Christie, it's definitely true. Her most famous stories (Death on the Nile, Murder on the Orient Express, And Then There Were None, and the Murder of Roger Ackroyd) often surprise modern readers with how thinly drawn the characters are. These stories are famous for their shocking, unpredictable endings. But other writers, such as Dorothy Sayers, did sometimes write with more depth, often in lesser-known stories.

For a very accessible read on some of this, see Lucy Worsley, "The Art of the English Murder." See also Julian Symons "Bloody Murder: From the Detective Story to the Crime Novel."