It rather depends on your perspective of what anarchism actually is, and what you mean by "anarchy as we would understand it today". The idea of "men with bombs" only applies to a specific radical mindset within anarchism. The thing is, anarchism is a spectrum, not one homogeneous entity. However, as it tends to happen with every movement, it has been stigmatized and stereotyped by the actions of the loudest individuals within the movement. It has occurred with conservativism, socialism, and with anarchism as well.

From a philosophical standpoint, the traditional mutualist anarchism proposed by Proudhon, who believed in peaceful social revolution, and Bakunin's collectivist anarchism, which advocated the violent eradication of the State by means of armed revolution (the aforementioned men with bombs), have evolved through the centuries.

You have anarcho-communists or ancoms or libertarian anarchists, who derive their conception of a decentralized society with respect for personal usage from Kropotkin's ideas. While collectivists believe in collective ownership and property, ancoms believe in the elimination of property, replacing it with the concept of free personal usage of communal objects and resources, based on individual needs. The basis for this theory is Kropotkin's The Conquest of Bread (1892).

Éliseé Reclus' green anarchism, a XIX century school of thought that gained traction during the XX, because it focuses in proposing peaceful alternatives to the environmental destruction capitalism creates. Currently, one of the most prominent figures within this framework is Layla AbdelRahim, who has written several essays and books critiquing domestication as the foundation of the current environmental and social crises, stating in Children’s Literature, Domestication, and Social Foundation: Narratives of Civilization and Wilderness (2015) that one of the main flaws in current education systems, is that they perceive children as 'wild' creatures that need to be tamed and domesticated, in order to transform then into agents that will in turn continue to domesticate both the environment and society to maintain the statu quo.

Anarcho-pacifism, proposed first by David Henry Thoreau and later by Lev (Leo) Tolstói, rejects physical violence and proposes instead civil disobedience as the means towards revolution. In 1848, Thoreau gave a speech and subsequently wrote an essay called Resistance to Civil Government proposing strikes, demonstrations, soldiers' desertion and not paying taxes as the way to stop funding and supporting a slaver's government.

Emma Goldman's anarcha-feminism, which proposes the abolition of the State as the means to erradicate the patriarchy. A prominent figure that advocated for feminist anarchism and polyamory in the early XX century, was Argentine activist América Scarfò.

Hakim Bey (Peter Wilson) and Todd May's post-anarchism or poststructuralist anarchism, rooted in poststructuralist philosophy, which breaks with most anarchist schools because it doesn't propose the abolition of the State, but rather the pursuit of a personal ethos, an individual ethics system that can allow a person to live up to their potential freedom, while doing the most good towards others and the environment. I recommend Bey's Avant Gardening: Ecological Struggle in the City & the World (1999) and May's The Political Philosophy of Poststructuralist Anarchism (1994).

These are only a few of the many schools of thought that exist within the anarchist ideals and principles, and as we can see, they are extremely diverse and more often that not removed from violent methods.

As for your question regarding Tolkien, philosopher David Bentley Hart wrote an interesting article in which he analyzes Tolkien's premise not as that of a violent anarchy but rather a mixture between classical anarchism such as Proudhon's, in which anarchy means simply the absence of a hierarchical and despotic government, with a sort of proto-ecologist approach, in which the figure of the monarch appears as a powerless one, with the only function of providing cultural cohesion to a society that ought to be organized communally, with decision-making placed in the hands of small communities, responsible for their own affairs and accountability.