Or was is all because of diseases?
To start with, people in the Middle Ages weren't dying of old age at 40. They died of lots of other stuff -- getting stabbed by clerks, getting crushed by barrels, drowning themselves in ponds, getting crushed to death by falling walls, plague, illness, blood infection, and so on. But these were generally the same ailments that were killing people at age 35 and that would go on to kill them in their 50s, allowing for differences in overall mobility and the work people were doing. People died by crime, through accidents, and due to illness and injury, sometimes due to illnesses and injuries they'd been nursing for a while before it ultimately overcame them. The idea that "people only lived to be 40", and that therefore a man of 40 would be considered a decrepit old man, is based in a misreading of historical figures of life expectancy that fails to account for the mechanics of infant mortality drawing down that average life expectancy considerably. If you survived infancy and childhood, and then subsequently lived to be 21, the odds of you living another 40 years or so were very good especially if you were noble and/or wealthy. People who lived to their 60s and 70s were not so rare as to be unheard-of, even in the lower classes.
However, living to be 65 isn't exactly prodigious old age in a 21st century modern-medicine context. How did medieval people conceptualize the stages of life and when people started to look old? We might look to medieval constructions of the "ages of man" -- three, four, six, seven, or even ten subdivisions of lifespan. These constructions generally recognized some intermediating stage of life between hale-and-hearty youth and fragile old age, sometimes in a quite fine-grained way. White hair and the decrepitude of age was surely not restricted to the chronologically elderly in reality (premature gray hair and diminished mobility are not modern inventions) but their use in these ages-of-man writings and artwork is as a sign of a life stage well past middle age -- a man in his 40s might still be considered in the prime of his life. I like Anne F. Harris' meditation on one ten-part "ages of man" wheel in the context of her own life and grief:
The general tenor surrounding old age in the medieval period is that of the ridiculous: the doddering, the wandering, the negligible. The medieval west was not a culture to respect its elders - there's no ancestor worship, no great amount of time or ritual spent making the old venerable. There's an interesting history there, no doubt. For now, these 10 Ages of Man fascinate. Here they are again:
You read it starting at 7 p.m. [on the clock-shaped arrangement of miniatures] - you are born and held by your mother warmly by the fire; you are young and vain; you measure things (a trade?); you are a young squire/knight falconing - ah, the time of love and war; at the noon position, you reign you are supreme you are self-possessed and full of possessions like a king; and then you bid farewell to yourself, walking away with a cane; you do in fact talk to small children (this roundel is so, so interesting - I can't see the writing enough to know, but will pursue the image to find out); and then you take to your bed and the physician reads your demise in the urine flask; a priest reads over your dead body, and (I think) empty sits the altar over which your mass was said. The scroll held by the king in the top left reads "Iuventus" (youth), the one held by the bearded prophet to the right "Senectus" (old age). The one to the bottom left reads something about infancy, I bet. The whole thing is in the shape of a wheel, and conjures up Fortune's wheel which has you up on top of the world, and down below getting crunched - all within a lifetime, sometimes within a day. I don't have to be lost in some unfixed state: this enormous narrative keeps moving, it has the momentum of history - and, as I've found out in the process of writing this post and visiting others' writings in the quest for images and answers to medieval questions, the companionship of kindred spirits.
Informal part 2: how did medieval people view men of forty?
Life-stages might be described by numerical age, in terms of physical capabilities, in terms of life-roles, or in terms of all of the above. Life might be divided into four stages -- childhood, youth, maturity, and old age -- or into three, eliminating the distinction between childhood and youth and widening maturity to encompass young adulthood. It might be defined further: Hippocrates gave seven stages: from birth to age 7, from 7-14, from 14-21, from 21-49, from 49-67, and from 67-death; Isidore of Seville in the 6th century gave the six stages as infancy (birth to age 7), childhood (age 6 to age 14), adolescence (14-28), youth (28-50), maturity (50-70), and old age (70+). These don't look quite like modern divisions, and even within certain historical periods like the Renaissance different authors don't necessarily use the same age divisions or even use the same words ("old age", "senectus", etc.) to mean the same spans of ages.
From the Pirkei Avot, a third-century CE text subsequently disseminated and discussed throughout the Middle Ages as a work of Jewish rabbinical ethics:
[Rabbi Yehuda ben Teima] used to say: At five years old one is ready for the scripture, at ten years for the Mishnah, at thirteen for the commandments, at fifteen for Talmud, at eighteen for marriage, at twenty for pursuit of righteousness, at thirty for full strength, at forty for discernment, at fifty for counsel, at sixty for old age, at seventy for gray hairs, at eighty for 'labors and sorrow' [Ps. 90:10], at ninety for decrepitude, at one hundred he is as though he were dead and had passed away and faded from the world.
(Taken from "The 'Ages Of Man' In Two Medieval Hebrew Poems" by Norman Roth)
This schema fits neatly into a hundred-year lifespan -- at 50 a man is at the height of his wisdom, at 100 he's nearly useless -- and I don't think we should actually take it to be a description of specific realities (surely not every man of forty actually is discerning!) -- this is how men should be, should be, and at the end likely will be even if they do all the preceding stages of life "right". Any given individual might in reality die during the course of any of these life-stages -- a five-year-old might drown, a twenty-year-old might be killed by a collapsing wall, a thirty-year-old man might be felled by respiratory disease, a man of fifty might commit suicide, and so on. For Augustine it was infantia, infancy, followed by pueritia, childhood, followed by adolescentia, adolescence, followed by juventus, youth or young adulthood, followed by gravitas, maturity, followed by senectus, old age; the Venerable Bede describes the ages of man in terms of their humoral qualities and adds a final category beyond just "old age", that of decrepitude. Writers like the author of The Parlement of the Thre Ages in the 14th century divide man's lifespan into (you guessed it) three ages, but also define these ages by their distinguishing qualities -- the rashness and luxuriousness of youth, the avarice and conceit of middle age, the quarrelsomeness and ugliness of old age. His representative of youth is a man of thirty, "Medill Elde" is a man of sixty. White hair rather than gray hair, the figurative "hoary head", is reserved for old age.
There's a clear image arising from all these conceptualizations of life stages that if unimpeded by other causes of death individual men and women could live to reach old age. However, actual men and women may not have had an expectation that they themselves would reach old age -- they hoped and planned to reach maturity, but people were abundantly aware of the possibility of sudden death before that point. These models of life stages served as a framework for understanding what tasks and roles were fitting for individuals of each age -- the idea of an age of legal responsibility, for instance an age at which one could be charged with a felony, or an age at which one could lawfully undertake certain actions, owes something to this periodization.
There was also a tension between the positive aspects of maturity and old age -- men were at the height of their strength and capability in maturity, at the height of their wisdom in (early) old age, but they were no longer young and beautiful as mature men and they were increasingly frail, dependent, and silly as they progressed into old age. The idea that elderly people might depend on others for their care and safety was a fearful thing, and respect for the wisdom of age seldom transferred into reliable respect for the dignity of aged individuals -- the idea of old men chattering or doddering away or walking with a cane or needing to be fed and nursed by others almost as if they were infants was counter to that same idea of wisdom and dignity, and it seems to have made medieval writers uncomfortable. This perceived helplessness of old people might also explain some of the cyclical depictions of the ages of men motif -- on top of a nice symmetry to the cycles of the zodiac and the cycles of the season, old people have effectively become like children again, as subject to the whims of their doctors and younger family members in old age as they were once subject to the whims of their mothers and nurses. It's an echo of the modern disgust at the thought of old people lacking teeth and wearing diapers, and that making them like children. Old people who sought to act and behave like young people in ways that were no longer within their physical capacity -- by doing things like behaving rashly and trying to physically exert themselves, or by falling in love with beautiful young women despite being rendered impotent -- were also seen as ridiculous, and as not "acting their age". Chaucer's Merchant's Tale evokes seasonal imagery again when he talks about the ludicrousness of a hoary old man, Januarie, trying to wed and bed a beautiful girl of nineteen, May. Similarly, old people who did not possess the wisdom of age were objects of scorn -- later in the 16th and 17th century, a common criticism of "under-churched" individuals who had failed to obtain an adequate religious education even over the course of 60 or 70 years of life was that they knew less than children might be expected to know.
At age forty, a medieval man was indisputably an adult, practiced in his trade, established in his family. He would likely be married, he would likely children of his own developing into manhood and womanhood in their own right. He would no longer be a pretty youth, but he still possessed procreative powers as well as lust. He would be near the tipping-point of physical development after which it's all downhill for men (in an abstract sense -- in reality some men reached this peak earlier and some later) but he wouldn't be so far past it yet, his willingness to go to war or to have sex might be economically unwise or unvirtuous but they wouldn't be absurd prospects. He would not yet be frail, or completely gray-haired. At age forty, a woman was also an adult, perhaps accomplished in her own trade as well as familiar with that of her husband's should she marry; if she was a nun, she might have risen into a position within her religious community. Her reproductive capacities might be nearing their end -- medieval constructions of menopause sometimes begin at 40, and multiple preceding childbirths took a significant toll on the body, sometimes a very obvious toll in the loss of teeth. There's something of a double standard there between women and men -- women being framed as likely to decay faster, and less likely to accumulate socially-dignified wisdom in their old age.
Some other reading:
Some other threads that might interest you on life expectancy and how people died: