These rates aren't always skewed by infant mortality. In the period I study, the 5-7th centuries in England, it's very rare to find infants' bodies in cemeteries (perhaps because their bones are so much more fragile and decompose more rapidly than children and adults; perhaps because infants were buried separately from the rest of the community until they reached a certain age). So when you encounter a cemetery with an average age at death of 25 (like several in Kent in the 6th and 7th centuries), that's not skewed by infant mortality - that's the average life expectancy for people who survived their first year.

In this period, it's common to see a fair number of young children dying, but there are also a decent number of dead teenagers and people in their 20s and 30s. It's rare to see many people in a cemetery who made it into their 50s.

Trusting this data relies on a few assumptions that are difficult to prove - it's possible that only important people were being buried in inhumation cemeteries, and people who die young tend to receive more attention from their communities (who are upset to lose someone before their time). Maybe people who lived past 50 were exposed, or sent down river, or had their ashes scattered, and so skew our sample by their absence.

It's hard to prove something from the absence of evidence, though, and looking at the data that is available, there's good reason to believe that - removing infant mortality from the equation entirely - people in 5-7th century England were, on average, lucky to make it into their 30s.

Surviving male skeletons show very high rates of arthritis, beginning at a very youung ages (early 20s), which suggests they were subject to hard physical labor. They were poor farmers, and they wore their bodies out through hard work at an early age. Some communities see women living longer - many of these older women had been better fed, and it seems like the community went to some effort to ensure they were healthy enough to make and care for children. In other communities, women didn't get this advantage, and we have a number of female graves containing very pregnant women who clearly died from complications related to childbirth.

If you survived childhood diseases, you could expect to be killed by hard labor, lung diseases caused by breathing woodsmoke in windowless houses every day, and childbirth at what we'd consider a tragically young age.

This improved during following centuries, but being a poor farmer is a hard life.

You are correct that a lot of these statistics are based on "life expectancy at birth" which is generally skewed very low because of high infant mortality rates. Generally, however, people who make it past infancy into late childhood have a much higher life expectancy, so it isn't even necessarily life expectancy assuming you live to 15 or 20, but much earlier. Six to ten years in old in some calculations I have seen based on skeletal evidence in the U.S. Southwest for Hohokam populations being the cutoff point where life expectancy goes up dramatically.

As to actual life expectancy, that is incredibly variable by time or place. Since you ask about London and medieval Europe, I'm afraid I can't help with that though I hope someone else chimes in with the answer you are looking for.