This section of The Road to Wigan Pier is essentially Orwell's musings on why socialism hasn't taken off as a political movement in the way that it 'should' have done. Here, Orwell is critiquing what he sees as dilettantes among British socialists - those attracted to communism and/or socialism because they seemed new, exciting and fashionable rather than genuine commitment to the cause. He is comparing them to what he sees as the other fads of the era, to be taken up and discarded according to the whims of the zeitgeist. I don't think that his contemporaries would see someone wearing sandals in the street and immediately think 'Oh look, a communist!', but equally they might well see a friend or family member's commitment to either esoteric footwear or communism as reflecting a phase, to be embraced then quickly discarded, in similar, slightly bemused terms. This, in Orwell's eyes, also contributed to the dismissal of socialist beliefs among many potential adherents as being a bit ridiculous, to be ignored by a serious person.

Implicit is the judgement that a true socialist movement needs to be built on more solid foundations, based on class-conciousness and true dedication to the ideals. A movement whose strength was based on its fashionability was fundamentally unable to achieve anything more than an ephemeral impact, and achieving true change requiring a long, dedicated struggle. Orwell's concern about the nature of individuals' attachment to socialism was not unique, but also reflected his own particular circumstances. The trope of the faddish, bourgeois socialist was only really prominent in the south-east of England, particularly in London and the universities, where socialism and communism found their way into fashionable literary and intellectual circles. Outside of the south-east, it was much rarer to see such a substantial middle-class involvement in socialist movements. Part of this can be explained by demographics - young, rich and hip types were most likely to live in or near London. It's also worth noting that in the industrial areas where such movements were most militant - such as Glasgow - middle-class populations were far more wary of working-class political movements, which meant that it was much more difficult to see one's involvement in fashonable terms.

Orwell wasn't entirely alone in this judgement. The Labour Party was often criticised for nominating middle-class candidates for too many seats, ahead of more legitimately working-class individuals such as trade unionists. The Communist Party of Great Britain, one of main sources of these criticisms, was frequently concerned in this period about whether its recruits and leaders were proletarian enough, even as it courted middle-class support in various ways. On the other side of the coin, MI5 was a bit too willing to write off involvement in socialist movements at university as representing a youthful phase that adherents had naturally grown out of, most (in)famously when it came to the Cambridge Five spy ring. This last point perhaps indicates a failing in Orwell's thinking here - some of these supposedly fickle adherents turned out to be some of the most dedicated to the cause over the course of their lives.

EDIT: Please check below for u/midnightrambulador's response, which includes the fuller version of the passage in question (as well as some others) and makes a good case that as relates to this specific passage, I'm overemphasising concern about the nature of middle-class adherence to socialism ahead of concern about alienation caused by socialists coming across as cranks.

I'll disagree a little with u/crrpit's answer in that I believe Orwell's critique was much more specifically targetted than just general fads or fashions. crrpit's absolutely correct that he's talking about a disconnect between middle class, bourgeois metropolitan and Southern socialists and the working class but capturing the tone of the class dynamics that Orwell is referring to requires context from earlier British socialist writers and thinkers. Specifically, Orwell mostly is not focused on dilettantes for whom socialism is new, fashionable or a phase. He does mention those in this form:

The typical Socialist is not, as tremulous old ladies imagine, a ferocious-looking working man with greasy overalls and a raucous voice. He is either a youthful snob-Bolshevik who in five years’ time will quite probably have made a wealthy marriage and been converted to Roman Catholicism...

But he gives much more focus not to those kind of fly by night believers as the problem but to committed 'socialists' that are middle-class progressives with the off-putting (to the masses and the working man in Orwell's opinion) tastes, education, snobbery and aesthetics of fringe middle class sub-cultures, and their class interests. He makes this clear in the way he continues the above sentence...

...or, still more typically, a prim little man with a white-collar job, usually a secret teetotaller and often with vegetarian leanings, with a history of Non-conformity behind him, and, above all, with a social position which he has no intention of forfeiting. This last type is surprisingly common in Socialist parties of every shade; it has perhaps been taken over en bloc from the old Liberal Party. In addition to this there is the horrible–the really disquieting–prevalence of cranks wherever Socialists are gathered together. One sometimes gets the impression that the mere words ’Socialism’ and ’Communism’ draw towards them with magnetic force every fruit-juice drinker, nudist, sandal-wearer, sex-maniac, Quaker, ’Nature Cure’ quack, pacifist, and feminist in England.

That sentence referring to the Liberal Party is an insight into the picture of a socialist he's painting. What ties together being a teetotaller, vegetarian or fruit eater, nudist/naturist, 'sex maniac', pacifist, Quaker, feminist, and 'nature cure' adherent is not that they were 'young, rich and hip' or fashionable but that they were markers that made up the complex of late 19th and early 20th century bourgeois liberal radicalism connected with social reform movements including the temperance movement, women's suffrage, and romantic and paternalist versions of socialism which Orwell relates to that of William Morris. Orwell's criticism is sort of targetted at Morris' extended circle, particular members especially, and their inheritors at the time that Orwell was writing. For example people like Henry Stephens Salt who advocated for reform of schools and prisons and was also the first writer to advocate explicitly for the rights of animals in the 1894 work Animals' Rights: Consideration in Relation to Social Progress, or Fabians like George Bernard Shaw who was also a vegetarian social reformer.

Or, and I think this is directly the person that Orwell is riffing on here, Edward Carpenter (died 1929 age 84) who was an advocate for homosexual rights, feminism, vegetarianism, trade unionism, teetotalism, anarchism, grow-your-ownism, nudism and was called the 'Saint in Sandals'. You can see a picture here from the Sheffield Libraries archive of a sandal template he made in 1895. They relate that he introduced them from India and quote him as saying "I soon found the joy of wearing them and after a little time I set about making them." Supposedly, this was the first successful introduction of sandals to Britain. You can also see in that gallery an ad showing that he would make sandals for people to order.

https://www.flickr.com/photos/shefflibraries/4929218755

Or as Orwell puts it in a letter to Jack Common from the same time (1936) as the writing of Road to Wigan Pier:

The trouble is that the socialist bourgeoisie, most of whom give me the creeps, will not be realistic and admit that there are a lot of working-class habits which they don't like and don't want to adopt. E.g. the typical middle-class socialist not only doesn't eat with his knife but is still slightly horrified by seeing a working man do so. And then so many of them are the sort of eunuch type with a vegetarian smell who go about spreading sweetness and light and have at the back of their minds a vision of the working class all TT, well washed behind the ears, readers of Edward Carpenter or some other pious sodomite and talking with BBC accents. (1)

Elsewhere in The Road to Wigan Pier we see the fruit juice mention again in direct relation to Morris and people like him.

The middle-class Socialist enthuses over the proletariat and runs ’summer schools’ where the proletarian and the repentant bourgeois are supposed to fall upon one another’s necks and be brothers for ever; and the bourgeois visitors come away saying how wonderful and inspiring it has all been (the proletarian ones come away saying something different). And then there is the outer-suburban creeping Jesus, a hangover from the William Morris period, but still surprisingly common, who goes about saying ’Why must we level down? Why not level up?’ and proposes to level the working class ’up’ (up to his own standard) by means of hygiene, fruit-juice, birth-control, poetry, etc.

So the people he's talking about aren't young anymore, they're mostly middle-aged to old, middle class, and from Orwell's view are cranks and weirdos espousing versions of progressivism and socialism that Orwell saw as old-fashioned or hang-overs from the C19th rather than new.

He makes this explicit in the picture he paints of two socialists as an example of what he's talking about:

One day this summer I was riding through Letchworth when the bus stopped and two dreadful-looking old men got on to it. They were both about sixty, both very short, pink, and chubby, and both hatless. One of them was obscenely bald, the other had long grey hair bobbed in the Lloyd George style. They were dressed in pistachio-coloured shirts and khaki shorts into which their huge bottoms were crammed so tightly that you could study every dimple. Their appearance created a mild stir of horror on top of the bus. The man next to me, a commercial traveller I should say, glanced at me, at them, and back again at me, and murmured ’Socialists’, as who should say, ’Red Indians’. He was probably right–the I.L.P. were holding their summer school at Letchworth. But the point is that to him, as an ordinary man, a crank meant a Socialist and a Socialist meant a crank. Any Socialist, he probably felt, could be counted on to have something eccentric about him. And some such notion seems to exist even among Socialists themselves. For instance, I have here a prospectus from another summer school which states its terms per week and then asks me to say ’whether my diet is ordinary or vegetarian’. They take it for granted, you see, that it is necessary to ask this question.

They're old men. Note the reference to Lloyd George, again a Liberal Party prime minister - out of fashion rather than current. The advent of educational institutions and schools pioneering vegetarian meals for children also comes out of the same turn of the century milieu of social reform. In his history of vegetarianism, Colin Spencer picks out schools like Abbotsholme which was founded in 1889 by Cecil Reddie, who had been influenced by Edward Carpenter, and "advocated diet reform and catered especially for vegetarian pupils". Spencer also makes the Quaker connection with Quaker schools such as St Christopher's at Letchworth where the headmistress added passages on vegetarianism added to Advice and Queries, a book of "principles suggested to all Quakers" (2)