My expertise is in the history of aesthetics as it applies to women's ideal beauty standards, so I'll speak to that. (edited to be more clear on what I actually know about. I can only tell you about the babes philosophers thought were ideal beauties, not regular ladies.)

This is really hard to answer, because the definition of "attractive" has varied widely throughout history and between cultures! Japanese historical beauty standards have been widely recorded, so that's a good case study in what I mean.

For example, this is a statue of Kichijoten, a Japanese goddess of Beauty (among other things) from the 12th century. This statue is from Heian-period Japan, during which plump women were considered very beautiful, and drawn-on eyebrows that slanted were in style. Earlier than this period, slim women were considered more beautiful.

During the same period in China, some of the same beauty standards reigned, with the significant difference that poetry describing beautiful women from this era in China frequently references red lips and white teeth, which is not mentioned in almost any Heian-period Japanese poetry. This is because at the time, Japanese beauty standards did not include white teeth, while Chinese beauty standards did. Instead, in The Tale of Genji and other 11th and 12th century Japanese literature, we have reference to blackening teeth as a beautification technique. In addition, while Japanese literature of this period references plump women as beautiful, Chinese poetry dating from the 900s on references slim women, and moves to a focus on plump women much more slowly than Japanese poetry.

Even between these two cultures, which interacted frequently in the 12th century (in complicated ways, which is a subject for a totally separate post), you'd get one group of people calling women with naturally white teeth attractive, while another group of people would be attracted to women whose teeth tended to discolor more easily. The same would happen with body type.

It's also important to note that beauty standards have changed significantly between 12th century Japan and modern Japan. Another useful touchstone period is the Meiji period in Japan, which lasted between 1868–1912. This is a bijin-ga, or picture of an idealized beautiful woman, from that period. She's become much more slim, and some historians say that this era of bijin-ga reflect increasing western influence in beauty. However, she still looks very different than the women shown in modern bijin-ga.

This study quantifies the specific differences in eye shape and size between Meiji-era beautiful women and modern ones. Again, we see a large difference over time in what traits are considered the most beautiful in women.

The above case study of culture-specific beauty standards that change over time is indicative of this tendency overall in our understandings of beauty. The history of beauty in specific cultures and regions is going to be complicated in each case, but this change will be a constant. If you'd like more detail on a specific culture, let me know and I'll see if it's one I can speak to.

There just isn't the kind of universal and long-lasting pressure to pick mates who look a specific way which would be required for any significant change in how people look over time in any particular direction.

(For more information specifically on historical Japanese and Chinese beauty standards, the book The Search for the Beautiful Woman : A Cultural History of Japanese and Chinese Beauty by Cho Kyo is a great start for non-historians. For more information on modern Japanese beauty standards, Beauty Up : Exploring Contemporary Japanese Body Aesthetics by Laura Miller is a nice introduction. If you're interested in the idea that beauty is culturally constructed, The Construction of Beauty: A Cross-Cultural Analysis of Women’s Magazine Advertising by By Katherine Frith, Ping Shaw, and Hong Cheng is a great article because it walks you through the way one might see if that's true in modern times, through really intuitive means. It's not a historical article exactly, but it explains the kinds of methods people use to make these points, without too much jargon.).

I can't speak to absolute levels of attractiveness--as /u/PlusTwoToCha pointed out, attractiveness is highly socially constructed and context dependent, and of course we don't have any photographs of 12th century women to compare modern women to. But I do want to take issue, from a biologist's point of view, with your assumption that attractiveness should generally increase over time. Hopefully this would be educational, and it should shine some light on one reason why you should be wary of evolutionary hypotheses for the historical change in a trait.

Suppose attractiveness has both genetic and environmental components (this means that how attractive someone is will depend on their genetics but also how they are raised and what stressors they are exposed to, and that how attractive someone is perceived to be will depend on some biological facts about the person viewing them, as well as the social/cultural environment). There is at least some reason to think that attractiveness does have a nonzero genetic component in both of these senses: standard, cross cultural markers of attractiveness, such as facial symmetry and certain body proportions, can be indicators of health and fertility (how effective they are as indicators is a matter of some debate). You might therefore expect this to increase over time, as sexual selection (and to some extent natural selection) favors it.

However, selection is not the only process that affects the genetic makeup of a population. Populations are also affected by mutations and genetic drift. Population geneticists usually define "fitness" in terms of the mean number of offspring an individual is likely to have. Most mutations that have an effect on fitness decrease it: if a trait is correlated with fitness, a mutation is much more likely to negatively affect it than to positively affect it (biological systems are generally well tuned, and a random change is much more likely to damage a system than to improve it). This is of course why selection is important: selection favors mutations that do increase fitness, which is why fitness doesn't simply decrease over time. Genetic drift refers to the fact that individuals sometimes have more or fewer offspring by luck of the draw. It's more prominent in smaller populations, and unlike selection, it doesn't favor beneficial mutations. So without natural selection, populations will experience many deleterious mutations which will not be purged over time, and, due to genetic drift, they can slowly spread to a high frequency in the population. All of this is encapsulated in an expression known as the Price equation, which measures the likely change in the value of a trait based on how correlated that trait is with fitness.

There is an argument to be made that an abundance of food, access to modern medicine, lower family sizes, etc. mean that natural selection has, in a sense, weakened over time. That is, the gradual relaxation in selection pressure means that there might not be any reason to expect a fitness-correlated trait to increase over time. Crow's argument in the linked paper is based on mutation accumulation experiments in fruit flies, where one fly is randomly selected to give rise to the next generation (so there is basically no natural selection and mutations accumulate unhindered by selection). In those experiments, fitness declines by about 1% per generation. This story comes with a number of caveats, some of which I outlined in an earlier comment on r/askscience. (Any "decline in fitness" in humans could be recast as a change in the environment, with the "decline" only being observable when comparing modern humans to some hypothetical ancestral environment.) This also comes with the caveat that increases in fitness can still be observed in recent modern history. The linked paper looked at Finnish church records and attempted to measure the variance in fitness, which is related to the rate at which fitness increases (that's called Fisher's "fundamental theorem of natural selection" and is a special case of the aforementioned Price equation). But at any rate, there does seem to be good reason to think that the strength of selection in humans has grown weaker, or (see last paragraph) that selection has substantially changed over time.

The other major issue is that the time scale over which selection works can be very long. Suppose a mutation increases fitness by about 1%, and suppose it's present in just enough individuals in a population that its behavior is determined by natural selection rather than genetic drift. Then its frequency is governed by a logistic function, and it will spread through a population in roughly 1/s*ln(Ns) generations, with N the population size and s the selective advantage, or the increase in mean offspring number (.01 in this case). Let's assume a time averaged population size of about 100 million people. That's about 1400 generations. Of course there will be many mutations affecting any given trait, but they'll generally have weaker effects. Natural selection is often very slow in humans, and cases where a single mutation has spread fairly quickly, such as lactase persistence, are the exception rather than the rule.

This all assumes a roughly constant environment, which in the case of human beauty would be to assume roughly constant beauty standards, or at least that people are attracted to particular genetically determined traits at about the same rate over time. But this study, which /u/PlusTwoToCha cited, further compounds the problem by arguing out that sexual selection on a particular trait has observably changed in recent time. Selection in a time-variable environment can be even slower and less efficient.