Common sense tells us that "lords" and commoners probably got along fine most of the time. If a "lord" was abusive to 'his' villagers, it would be unlikely that he would get much work or much quality work from them.

The lord relied on the commoners to farm, to build his castle, to aid in defense (in some Medieval periods and places). The ability of the commoners to make the lord poor and his life miserable, even without going anywhere near such radical action as revolt, was high.

A wise lord would probably treat the commoners well. Of course, then as now, not all people are wise.

Well, here's what Machiavelli, looking at feudalism at the tail end of the middle ages proper, had to say about lords, or as he calls them "gentlemen,"

"And to explain more clearly what is meant by the term gentlemen, I say that those are called gentlemen who live idly upon the proceeds of their extensive possessions, without devoting themselves to agriculture or any other useful pursuit to gain a living. Such men are pernicious to any country or republic; but more pernicious even than these are such as have, besides their other possessions, castles which they command, and subjects who obey them.

This class of men abound in the kingdom of Naples, in the Roman territory, in the Romagna, and in Lombardy; whence it is that no republic has ever been able to exist in those countries, nor have they been able to preserve any regular political existence, for that class of men are everywhere enemies of all civil government. And to attempt the establishment of a republic in a country so constituted would be impossible.

"The only way to establish any kind of order there is to found a monarchical government; for where the body of the people is so thoroughly corrupt that the laws are powerless for restraint, it becomes necessary to establish some superior power which, with a royal hand, and with full and absolute powers, may put a curb upon the excessive ambition and corruption of the powerful."

Machiavelli saw lords as the inveterate enemies of liberty and equality; addicted to warfare and infighting and incapable of an honest day's work. He wrote that the only way to keep order, peace and liberty in a republic was that if any citizens find a noble, "they kill them, as being the chief promoters of all corruption and troubles."

I would say that while it's true that nobles had no interest in working peasants to death, and while the horror stories of Prima Nocta or Evil Prince John are overblown or entirely false, the feudal system was basically founded upon constant internal warfare and infighting and upon the economic, political and cultural subordination of almost everybody to a handful of people. Lords were full time warriors, and they saw their holdings as revenue generators to allow them to keep fighting. The extreme localization and decentralization inherent to having multiple powerful aristocrats created constant internal warfare that was a devastating factor that common people had zero influence over. Republicans like Machiavelli preferred to stay as far away from lords and possible behind the walls of their cities.

So no, lords were not purposeful unjust or abusive, but they did see peasants as little more than revenue machines fueling their own internal political and military squabbles. They also consumed an enormous amount of the wealth of society while producing nothing but the dubious promise of military protection (from wars caused by the internal conflict inherent in feudalism)