Some people did take Hitler's threats quite seriously. There were plenty in the Soviet Union who were aware of Hitler's stated intentions and the nature of his regime. Most notably, Bukharin, always the most Western-looking of the Bolshevik leadership, was familiar both with Mein Kampf and how it sat within the increasing corpus of 'race theory' that was gaining ground in Europe. From 1933 to 1936 he warned in print against the "frantic anti-Semitism and incredible contempt for the colonial peoples" of the German leadership.

Nor was he blind as to what this meant in real terms. I can't find a complete copy now but his speech at the 1934 Party Congress included a sharp passage on this subject. He spoke openly of fascism as "open robbery, a frankly bestial philosophy, the blood-stained dagger and an open era of throat-cutting". Crucially, and again referencing Mein Kampf, he understood how Hitler "openly called for the destruction of our country" and went on to blackly comment that:

Hitler wants to drive us into Siberia and the Japanese imperialists state that they want to drive us out of Siberia, so that probably the entire population of our country would have to be located on one of the blast furnaces of Magnitogorsk!

In this Bukharin was prescient but he was hardly alone in being aware of Hitler's public programme for the East or his regime's virulent anti-Bolshevism. A year later, at the 1935 7th Congress of Soviets, Molotov also warned of the threat from Germany, directly referencing Mein Kampf. Litvinov, the key anti-Nazi figure in Moscow, was also fond of quoting Hitler to support his position. There was no doubt that the Soviet leadership was aware of Hitler's words and stated ambitions. This was a major driver of the increasing rearmament programme that kicked off in the mid-1930s.

Without going into a full history of the interwar diplomacy, the Soviet leadership therefore pursued a policy of collective security designed to contain Nazi aggression. This was hindered by the mutual suspicion between Moscow and Paris/London, the need to protect both flanks (ie the threat of Japan) and the possibility of resurrecting trade with Germany (particularly in weapons technology). But generally the USSR was formally hostile to Nazi Germany during the 1930s, hence the shock of the 1939 Non-Aggression Pact. Whether flawed or not, the hostile Soviet approach to the Nazis was informed in no small part by Hitler's words and stated intentions.

Sources:

For a broader picture of the international tensions, plus the quote from Bukharin, see Davies' Years of Progress. The latter is also covered in Cohen's Bukharin, which notes the anti-fascist campaign that he led in the press.

British intelligence was aware of the threat that Nazi Germany posed but was very slow to get to grips with Hitler and the Nazi apparatus. Nazi Germany had been identified as Britain's 'ultimate enemy' in 1934 and there were some MI5 officers who had read Mein Kampf and did take it seriously, but at the time they were desk officers and no one was listening in Whitehall. Additional resources to tackle Nazi Germany were not forthcoming so intelligence agencies continued to spend the bulk of their time and resources on the Comintern. At the outbreak of war MI5 had no functioning understanding of either the Nazi political structure or its intelligence capabilities. When historian Hugh Trevor-Roper joined SIS (MI6) in 1941 he was astonished to find that none of his colleagues had read Mein Kampf or any other Nazi political writings, though SIS would shortly hire an astrologer and a water diviner to gain insights into Hitler's decision-making.

Source:

Calder Walton, Empire of Secrets: British Intelligence, the Cold War and the Twilight of Empire, 2013

Can I just add as a side note that the Allies did know of the existence of Mein Kampf and the threat it might have caused however it still needed to be translated into English, so a man named "Edgar Dugdale" from 1931 - 1933 worked on translating the first English and abridged edition, which can be found to buy here. This is an American copy that's why it says "My Battle" instead of "My Struggle" like the English copies.

Dugdale wanted to release his translation in early 1933, however a local Nazi representative (Because this was in the UK that representative could've well been part of Oswald Mosely's "British Union of Fascists" as well, but don't quote me on that) wanted the translation to be abridged further so publication was pushed back until October '33.

However the Dugdale translation wasn't well received and quite a bit shorter than the original German edition, so between 1936-7 a guy called "James Vincent Murphy" retranslated it, this edition now known as "The Official Nazi Edition" wasn't published until 1938, so that might be able to explain why not that many people read Mein Kampf before '38.

There is still dispute over the fact that even popular supposed unabridged versions like Murphy's are abridged to an extent and so there are newer translations still, trying ever more to decipher more of what Hitler was saying than those previous.

Sources and Further Reading:

Mein Kampf, Original Edition

Mein Kampf, Dugdale Translation

Mein Kampf "Official Nazi Edition", Murphy Translation

Mein Kampf, Jaico Translation

Mein Kampf: Analysis and Summary

"Why Did My Grandfather Translate Mein Kampf?" ~ BBC News

Actually, one person in literature/criticism took Mein Kampf quite seriously.

Kenneth Burke, a major figure in humanities in the 20th C., did an analysis of MK, called "The Rhetoric of Hitler's 'Battle.'" In it, Burke states very clearly that Hitler's arguments very clearly lead to a disastrous outcome, not only for Jews, but for the world.

MK flopped because it was, as you note, long and boring. However, HAD others read MK with a critical eye, or had taken Burke's insight more seriously, there may have been a different outcome.