Were there not moments where a crusader poured boiling oil over someone and thought “Oh shit that’s sorta inhumane” or a Mongol soldier looked at a massive pyramid of skulls and thought “Crap I‘m not too proud of that”. There seems to have been so much brutality in history, so did it really take until the 20th century for people to come up with things which were considered unacceptable?
The short answer is: no, not at all.
But the longer answer is that while something we could call 'morality' has always existed, how we conceive of morality has changed a lot.
Specifically when it comes to humaneness, even in ancient Chinese philosophy, there is the concept ren, which we could plausibly translate as 'humaneness', and ren is a chief ethical virtue.
That being said, for the rest of my answer I will restrict myself to the history of Western philosophy just because it will make it simpler to answer the question.
In ancient Greece, we find the idea of virtue (arete). Virtually every philosopher after Socrates (died in 399 BC) wrote extensively on this idea, and we can get a sense from how much these philosophers talked about it and what they had to say that ancient Greek cultures were deeply driven by a sense of virtue. They employed many of the same concepts we do to talk about virtue, especially shame and honor. A list of Greek virtues might not feature honesty or the sort of warmth and friendliness that we think are virtues, but courage, self-control, justice, wisdom, etc., would appear, as well as some not-so secular ones that are still familiar to us, such as piety. We sometimes see Socrates depicted in conversations in Plato's dialogues as appealing to an interlocutor's shame, so that feeling of 'oh, man, this is sort of evil' that you wonder whether could belong to a crusader undoubtedly did belong in the pre-modern person's "moral vocabulary", so to speak.
Of course, there were bad people then and now, and sometimes people are shameless and sometimes people are not.
But more to the point, older civilizations had just a very different conception of what particular things were right and wrong, and I think this explains what you are asking about too. (Although I definitely think some part of the explanation does have to state that if we judge whether a society "had morality" by its worst examples, then we'd probably say our own society "doesn't have" morality -- which is absurd since our society obviously does, and therefore we shouldn't judge whether a society "has" morality by its worst examples.) Ancient Greeks, for instance, would expose babies in various circumstances -- this means that they'd leave them out in the wilderness to die. Sometimes this was done for reasons of disability. You can see an example in Sophocles' Oedipus, where Oedipus is exposed due to a very worrisome prophecy. Plato seems to support exposure of children in cases where the child is an offspring of two parents who have bad characters in the Republic. (If this is really his view in the Republic, then he walks it back in the Timaeus and says that children should merely be sent away to live in another city in these cases.) Aristotle definitely is in favor of exposure in cases of disability, in the Politics.
If you look at exposure with "modern eyes," you might think that morality was invented only in modernity, but you'll miss the fact that these societies did have a conception of morality, but they thought that exposure was morally permissible (and Plato certainly makes it sound morally required).
However, modern ethics does represent a real change from ancient and medieval (Western) ethics. In antiquity and medieval philosophy, morality was not impartial. What I mean by this is that a lot of your moral responsibilities (i.e., what you should do and what you should not do) reflected who you were and who was close to: you might, for instance, have a special duty to yourself, to your friends, and family. This doesn't mean that you could wantonly murder complete strangers, but you'd be expected to do different things to/for different people. In much of modern ethics (especially strands emerging from 19th-century German and British philosophy), you have to treat everyone impartially: that means there's no moral reason why you should treat yourself, your friends, and your family different from how you treat strangers.
This leads to many of today's philosophers arguing things like that we should give up as much money to poor people as we can without severely damaging our own well-being. For instance, you can see Peter Singer's famous essay, "Famine, Affluence, and Morality", where he says that:
"It makes no moral difference whether the person I can help is a neighbor's child ten yards away from me or a Bengali whose name I shall never know, ten thousand miles away. [...] The moral point of view requires us to look beyond the interests of our own society. Previously, [...] this may hardly have been feasible, but it is quite feasible now. From the moral point of view, the prevention of the starvation of millions of people outside our society must be considered at least as pressing as the upholding of property norms within our society."
Maybe if we tried to press ancient Greek ethics really hard, they'd also find some moral responsibility to help very distant strangers, but they'd certainly never say that it literally makes no difference whether it is a neighbor or a very distant stranger. There'd always be a strong attachment to one's own community.
We can also compare responses to the famous so-called trolley problem. You are walking down a trolley track. Five people are tied down on the track, and a trolley is barreling towards the group. You can flip a lever, and the trolley will barrel down to the other track, but there is one person tied down there. Should you intervene and flip the lever, or not? Today's ethical theories debate at great length what you should do. Some say you should flip the level. Some say not to. But let's say the person tied down on the track was your mother. Every major modern ethical theory would say that this doesn't make a moral difference. This doesn't bear on your decision at all, and it isn't a moral consideration. But certainly every ancient moral theory would say it would be shameful for you to let your mother die or even kill her!
Many readers of Plato are struck by the drama in his dialogue the Euthyphro. Euthyphro is on his way to court. His father has just killed a slave, who himself has killed another slave. Euthyphro believes that his father's homicide was shameful, so he is taking his father to court. Socrates is actually aghast at what is being done by Euthyphro! He thinks that Euthyphro's behavior of prosecuting his own father is shameful. Today's students, when I have taught them this text, are always on Euthyphro's side, but there just was a different approach to impartiality in antiquity and in the medieval period. The 19th-century ethical theories from disparate parts of Europe changed this.
The impartiality of today's ethical theories sometimes doesn't sit well with us when we aren't in a philosophy classroom. ("Of course," you may say, "I should treat my mother differently than I treat a complete stranger.") But philosophers like Peter Singer believe that this is a defect of how we normally think about morality -- it reflects our own self-centeredness that we tend to weight the value of ourselves and people close to us more highly than we weight the value of strangers. For Peter Singer and others, good moral reasoning is consistent: they think that there is no good, principled reason why we value some people more than others.
Pre-modern philosophers naturally disagree. Thankfully when you walk into a philosophy classroom today, good teachers design courses that pull from various traditions, such that students often walk out of a philosophy class with knowledge of ancient and modern theories.