Did the West believe that only the East (China, Japan, Korea) could challenge them as a civilization? Because I've never heard of a brown or black peril before.
Le péril jaune
«Partout où l'ouvrier chinois ou même nègre est en concurrence avec l'ouvrier blanc, dit M. E. Faguet, celui-ci est vaincu». Nous avons assez vu cela dans l'Outre-Mer de Bourget, où le terrible problème des races est si nettement posé! «L'ouvrier à cinq sous est naturellement vainqueur de l'ouvrier à cinq francs».
The Yellow Peril
"Wherever the Chinese or even the Negro worker is in competition with the white worker," says M. E. Faguet, "the latter is defeated". We have seen enough of this in the Overseas Territories of Bourget, where the terrible problem of races is so clearly posed! "The worker at five sous is naturally the winner of the worker at five francs."
While Europeans did recognize differences of race and ethnicity based on skin color and other perceived gross physiognomic differences, the idea of "color" in the popular sense was categorized and popularized by Carl Linneaus in Systema Naturae (1767) where four great varieties of humanity were recognized and assigned colors according to the old "four temperaments" theory (sanguine, choleric, melancholic, and phlegmatic). Europeanus was white (sanguine), Americanus was red (choleric), Africanus black (phlegmatic), and Asiaticus was yellow (melancholic). By the late 19th century, the term "Yellow" as a descriptor for all peoples from Asia was in common parlance.
Novikow's polemic against Asian workers was representative of economic concerns that European labor would be undercut by cheaper labor from abroad; this is a familiar and perennial argument to support racism, discrimination, and even slavery, and was not uniquely directed against Asians. We might draw a parallel with the American Civil War:
Be assured that emancipation itself would not satisfy these fanatics: that gained, the next step would be to raise the negroes to a social and political equality with the whites; and that being effected, we would soon find the present condition of the two races reversed. They and their northern allies would be the masters, and we the slaves; the condition of the white race in the British West India Islands, bad as it is, would be happiness to ours. There the mother country is interested in sustaining the supremacy of the European race.
Military opinion of Asian countries dropped in the 19th century, after the defeat of China during the first Opium War (1839-1842) and the second Opium War (1856-1860), and Commodore Perry forcing open Japan in 1852-1854 - and there were numerous other conflicts between American and European powers where gunboat diplomacy reduced independent states, forced open trade, and established colonies.
At the turn of the century, however, sentiments were changing. After the stunning victory of Japan in the Russo-Japanese War (1904-1905), Kaiser Wilhelm II borrowed Novikow's term and began to agitate on the immense menace posed by Asia as an aggressive militaristic enemy utterly opposed to Europeans on racial grounds.
"Future war" stories were popular, and the idea of possible national and global conflict on racial grounds entered the popular imagination through works like The Yellow Danger; Or, what Might Happen in the Division of the Chinese Empire Should Estrange all European Countries (1898) by M. P. Shiel, La Guerre infernale *1908) by Pierre Giffard, "The Unparalleld Invasion" (1910) by Jack London, the Fu Manchu novels of Sax Rohmer beginning in 1913, and "Polaris" (1920) by H. P. Lovecraft.
It has to be remembered that especially in the United States this kind of "race war" literature had long been a part of the popular culture - both in terms of Colonial, frontier, and "Western" fiction which pitted "white" Europeans and Americans against "red" Native Americans, and slave revolt and Reconstruction narratives that focused on possible uprisings of Black Americans against "whites" - most notably The Clansman: A Historical Romance of the Ku Klux Klan (1905) by Thomas Dixon Jr., which would go on to inspire the stage play and then the film The Birth of a Nation (1915), which in turn led to the founding of the second incarnation of the KKK.
Like the KKK, the rising tide of popular consciousness about the "Yellow Peril" (also "yellow terror," "yellow menace," etc.) led to genuine racial discrimination and political action. The Immigration Act of 1917, also known as the Asiatic Barred Zone Act, specifically attempted to bar further immigration into the United States from China, Japan, and other countries in Asia on racial grounds. The rising power of Japan in particular after WWI made them a player in the Washington Naval Conference (1921-1922) - and they resisted the efforts to limit the expansion of their power and influence in the Pacific. The collision of interests would lead to many prognosticating war between Japan and the United States long before the attack on Pearl Harbor in 1941.
There was a contemporaneous "Black Peril" scare, but the issues involved were usually either uniquely colonialist (as in South Africa), or more accurately a fear of rebellion from local "black" populations in the Americas, either in post-Reconstruction the United States, or during the American occupation of Haiti (1915-1934) and the Dominican Republic (1916-1924), although there was "future war" fiction which imagined the possibility of a Black Peril-style conflict on racial grounds.