At the bottom of every Google Ngrams result, you can see that it breaks the hits down by year ranges. If you click them, it'll show you what's causing the hits. In this case, there was a book in 1851 that categorized the animals of the "East Indies" that has an entry on the Bruh, apparently a local term for the southern pig-tailed macaque in Indonesia. There are also, as with all Google Ngrams results for short words, a lot of "false hits"; when you see that kind of "jerky" result it also is usually an indication that the sample size is pretty small — the word "bruh" was barely used at any time, and always small relative to all other words, so an increase in three or four citations means that it suddenly looks very outsized compared to the whole curve.

So, no, nobody was going around calling each other "bruh," unless there were some people calling one another monkeys in Indonesia. What you're seeing is mostly a statistical artifact.