You are looking for what is known as an explanation along the lines of euhemerism; that is, the description of the "actual basis" of a widespread motif in oral tradition. Finding the "real thing" behind a folk belief is usually not a productive exercise, although it has been pursued at least since the 4th century BC when Euhemerus advanced this notion. And it continues to be a popular idea with many people.

Belief in elves and related supernatural beings extends from Ireland to Sweden and Northern German to Iceland (with virtually everywhere in between). It is not reasonable to conclude that all of these people throughout all the centuries of belief were extrapolating the existence of supernatural beings based on how some people looked. In addition, although these supernatural beings figure in shared legends and folktales and behave in much the same ways, their appearance in the various cultures can be different: some are short; some are he size of normal people; and some can be larger. Some are beautiful while some are ugly. Some have hallow backs or cow tails. But they all behave in generally the same way and appear in historically related legends and folktales.

This is not to say that real-life observations have not reinforced belief and supported existing legends. For example, stories that focus on the presumed interest of these supernatural beings in abducting babies is widespread. The legend in question features a changeling, one of the supernatural beings transformed to appear like the abducted baby. But the replacement fails to thrive. In the legend, the mother either mistreats her presumed offspring or does something astonishing in front of the baby. In either case, the baby usually confesses its true nature and a supernatural being appears with the human infant to reverse the exchange, bringing things to right. There has been a great deal of focus on how babies who failed to thrive may have encouraged the legends about changelings. In my work on Cornish folklore, for example, I was able to identify one clear expression of this legend type, but there were many more accounts of infants who were mistreated because they were suspect, and yet no supernatural being appeared with the "real" human infant. These tragic instances no doubt reinforced the folk belief, but they cannot be regarded as the actual basis in the belief in elves in general.