There is no straight-forward answer to this question, certainly not from the historian's standpoint. Furthermore, this question is problematic for several reasons. It has certain suppositions that I think are important to dispel or nuance.
Firstly, feudal-like systems of government existed and governed societies and its people successfully and less successfully for hundreds of years. Far longer than modern democratic modes of government have been around (again with varying degrees of 'success'). Saying it just didn't work at all is ignoring the multitude of political, social and economic systems of many centuries, and the successes (and of course failures as well) these feudal-like systems had with dealing with the circumstances.
Secondly, capitalism (and I would argue communism as well) is mostly an economic system. The focus lies on markets, property ownership and economic restrictions and liberties. Feudalism was certainly not strictly focused on pure economics. It was a broad social system. Military, political and economic relations were all intertwined in different ways than in our modern world. Feudalism encompassed social, military, legal, moral, even religious customs and relations between different people and classes in the medieval society. These are areas of society modern capitalism generally has little to do with. Putting capitalism/communism as a successor to feudalism seems to me to be fundamentally flawed in this regard.
Thirdly, both capitalism and communism on the one hand, and feudalism on the other, are not monolithic or uniform systems. The U.S. economic policy is not the same as Belgium's economic policy, or Russia's, whilst all three are democratic countries with (degrees of) economic freedom. China's economy or its society is not the same as it was in Mao's time, although it's still ruled by the communist party. And this is the same for feudal societies. The political and societal situation or system of government in 12th century France was not the same as it was in The Holy Roman Empire or the Spanish kingdoms.
And finally, maybe most importantly, concepts such as natural evolution or historical inevitability are topics that are extremely problematic and for a large part outside the field of history. Unlike physics, chemistry or any of the natural sciences, history cannot be repeated or subject to experiments. Scientists talk meaningfully about natural laws, inevitable (and calculable!) reactions exactly because they can test out their ideas and theories, again and again. With history there is just one go, no do-overs, no going back or trying something again. Although history has its own methodology to create historical knowledge, questions such as 'is one historical state the only natural evolution of a previous historical state?' are outside its scope. Therefore historians generally do not make bold claims that certain historical events were inevitable, or historical developments were a natural evolution. It is after all impossible to conclusively demonstrate that something couldn't possibly have gone another way. Often it even seems as if historical people or events had all the odds stacked against them and the historical outcome was a something of a fluke.
Another thing to remember is that ideas that posit certain historical developments as natural tend to have underlying political or moral motives. A point eloquently argued by Karl Popper in The Poverty of Historicism. For instance, the germanic domination of Europe was seen as a natural evolution by the nazis. Marx claimed the proletarian revolution was inevitable. Hegel saw in the reign of Frederik III of Prussia the natural culmination of the inevitable path of history, from antiquity through the middle ages down to his own 'enlightened age'. These are all examples of answers to similar questions such as the OP asked. And it's clear they come from a specific ideological place.
But just because historians do not explicitly answer these types of questions, for all the reasons mentioned above, does not mean nobody else tries their hand at them. Questions concerning historical patterns, necessities, (natural) evolutions, etcetera are actually specifically tackled in the discipline of philosophy of history. And since it is philosophy, it should come as no surprise that there is a lot less consensus than one might hope for. The aforementioned Karl Popper argued history is in essence unpredictable. Claiming the evolution of political or economic systems as inevitable or natural is obviously problematic from this perspective. The evolutionary biologist Stephen Jay Gould explained the related point of the fundamental contingent nature of history and evolution itself in Wonderful Life: The Burgess Shale and the Nature of History. Francis Fukuyama took off from Hegel and Marx and argued that our modern democratic and capitalist society is the natural endpoint of history in The End of History and the Last Man. These are engaging ideas, and I highly recommend reading some of them. Nevertheless, none of these ideas are uncontroversial. There is just no easy answer to these kinds of questions.
My own little observation or remark that might help put this question into perspective, is that, even though some of these theories mentioned above are certainly interesting or even appear to be plausible, it is important not to look at our own time as the endpoint of history. I'm sure the brightest minds in medieval Europe were absolutely 100% positive that christianity was the endpoint in history. How could anything possibly replace the literal word of god?! Well, as we now know, stuff happened. And even though democracy and capitalism might seem to us as a natural evolution or a natural state, maybe even an endpoint to that evolution, nobody can foresee the future, and history is fraught with unexpected twists and turns.
Sources:
The Poverty of Historicism by K. Popper
Wonderful Life: The Burgess Shale and the Nature of History by S. J. Gould
The End of History and the Last Man by F. Fukuyama
Philosophy of History. An Introduction by W. H. Walsh (specifically the first two chapters tackle the problems addressed in this thread, and the chapters on speculative philosophy of history provide examples of ways these questions have been dealt with in the past).
Maybe Feudalism is just an inadequate model for understanding the Middle Ages. /u/idjet and /u/TheGreenReaper7 take a very dim view of Feudalism as a historical concept. See the subreddit FAQ for links to some of their greatest hits on the theme.