There is a book (not translated in English) called "Memoirs of Stalin's private secretary". The author's name is Boris Bazhanov - he was a secretary of the Politburo, and a personal secretary to Stalin, from the beginning of 1920s to 1928, after which he defected to Persia, and then to France.
I've gotten to read his book as part of a larger effort to find historic evidence for and against social reform not happening organically, but being enacted and stimulated by governments "from up top". There's a passage in the book that seems to be exactly what I want (text below is my translation from Russian) - allegedly, this is what Lenin's personal secretaries told Bazhanov when he asked them if Lenin actually believed in the New Economic Policy.
"Lenin told them: Of course we did fail. We wanted for a new communist society to appear magically by itself, while it turned out to be a question of decades and generations. So that the Party does not lose its soul, its faith and willingness to fight we must behave as if this return to market economics, to capitalism, is some kind of temporary fallback. But we ourselves must see that our attempt has failed, that people's psychology, that age-old skills of life that they have accumulated cannot be broken in a snap. We could try and force the people into our new society by starting a war, but it's not certain if we'd even retain power in such a meat grinder."
As I've said, this seems to be exactly what I want - evidence of Lenin himself admitting that enforcing social reform did not work out. But Bazhanov's book is full of episodes where he portrays himself as the only sane and moral man in the whole of Politburo, details Stalin as some inhuman power-hungry monstrosity (expectedly - Stalin did order him killed once he defected), and reads a bit fiction-like at times. I'm not willing to take his information without a good pinch of salt.
That is what constitutes my question. Are there other sources that confirm he did in fact hold the view that the quote says he did?
Conveniently the website Marxists.org has an archive of Lenin's writings and speeches. Including the following article where Lenin discusses the New Economic Policy.
V.I. Lenin: The New Economic Policy And The Tasks Of The Political Education Departments, Report To The Second All-Russia Congress Of Political Education Departments October 17, 1921, https://www.marxists.org/archive/lenin/works/1921/oct/17.htm
In this article Lenin specifically calls War Communism, the economic policies his government had pursued, a mistake. And not just a mistake in hindsight, but one that Lenin says the Bolsheviks anticipated:
Ever since 1917, when the problem of taking power arose and the Bolsheviks explained it to the whole people, our theoretical literature has been definitely stressing the necessity for a prolonged, complex transition through socialist accounting and control from capitalist society (and the less developed it is the longer the transition will take) to even one of the approaches to communist society.
Lenin positions this decision and the decision to introduce the NEP, as a retreat like the Communists underwent at times during the civil war against the White Russians - the counter-revolutionary forces. Since the Communists won the civil war in the end, the implication of the comparison is that they will win again in the end in the economic front. But Lenin, a bit like Churchill in WWII, positions this as a long hard road:
Of course, tasks on the economic front are much more difficult than tasks on the war front
...
And the substance is that the fight will be even more desperate and fiercer than the fight we waged against Kolchak and Denikin.
And then he positions it as a heroic battle:
the world has never witnessed a struggle like the one we are waging now
It's a strong piece of rhetoric. I recommend reading the whole thing.