You can blame cinema for this.
As far back as one of the first films ever, Edison's adaptation of Frankenstein, we see our creature as a pathetic sidekick and jealous simpleton.
However, the most popular Frankenstein was 1931's classic Universal version starring Boris Karloff. Popular culture has made this the very symbol of the creature: look at lunch boxes, cheap Halloween candies and decorations, parodies, toys, anything that features "The Creature" and you'll see him green with stitching around his temple and iron studs in his neck; much like the ruby slippers of 1939 Wizard of Oz, these would become the prototypical vision of what Frankenstein's creature (or Frankenstein himself in many people's minds) has become.
Since most adaptations of Frankenstein leave off the "twist" in the narrative's ending, the creature's soliloquy towards his dying master, intelligent and compassionate despite his mental illness, we only see the monster. I suppose audiences love the spectacle of the poor scientist and his blushing bride being attacked by a demon created with the "wrong brain" instead of the scientist as monster, his creature only hating it's own existence.
Here's where your common portrayal of the Creature began: https://www.youtube.com/watch?v=AkSbwiKP3mo
Edit: please forgive me that I could not find enough viable information on early stage productions of Frankenstein, which is actually where the perception of the creature as a mute brute began as early as the 1820's. I found some sources after a bit more research (http://members.aon.at/frankenstein/frankenstein-theatre.htm). From "Presumption; or the Fate of Frankenstein" from 1823:
"But the most significant changes are the omission of the Walton subplot and - even more important - the muteness of the Monster. Peake made it a brutish creature with an infant's mind and unable to speak. It does not develop human emotions and is only capable of rage and violence."
The reason I did not press this much is because the modern day impression of the creature as the brute is definitely the James Whale adaptation from cinema. The silent monster has existed for over a hundred years prior however, but as I said before the green man played by Boris Karloff, in most people's minds these days, I would speculate is their impression of the Creature.
It more about how to portray a murderer. In the novel the monster strangles a small boy and frames a girl for the murder so she will hang. He kills without compunction and is quite capable of understanding his actions. In the clip from the 1931 film below, he kills a girl accidentally and is horrified at what he did. The monster in the films is like massively strong primitive who is unable to understand what is around him but has a genuine sadness to him. The monster in the book is a genius and erudite but without a shred of humanity or remorse. One has a mind and no soul, and one has no mind but a soul. It's easier to portray one of these than the other.
The New York Public Library actually has a great collection of resources dealing with this topic here - Frankenstein. It describes several of the first English stage productions that haven't been mentioned here yet. As the book was reworked for theater and film productions, the character was changed for many reasons, but the part of the Denlinger essay that I found most interesting was that
Once abstracted from the story, the Creature becomes shorthand for anything overgrown, misshapen or threatening. In 19th-century Britain, the Creature was put to the service of racist or xenophobic purposes.
Related question: When/why did the name "Frankenstein" begin to refer to the monster instead of its creator?