There are obviously many different facets to this answer, so I'll try to start from the broadest aspect. First, the Renaissance didn't really affect the common man of Europe until the Enlightenment. While the Borgias and the Medicis may have proven Machiavellian virtu, paper and art were still luxuries for laymen until much later on. It's only around the time of Charles Perrault, Hans Christian Andersen and the Brothers Grimm that we see a consolidation of stories that lived throughout time through spoken word. In regards to specifically folk songs and stories, it's important to remember that prior to the Reformation, music was seen as a medium only used to praise the glory of God, not to reflect the beauty of humanity. The Catholic Church was so weary of music's potential for "baser" inclinations (ie suggestive or passionate sounds) that they banned tritone harmonies for a time. After the establishment of Protestantism, music became more secular. Over time, music adapted to the more common ear, a far cry from Thomas Tallis, but more broadly enjoyed.

Most of the reason that we don't hear about many folk tales or songs from the Middle Ages is purely due to accessibility. Middle English is difficult to understand and often serves more of a religious purpose than literary, and Old English is impossible for the basic Modern English speaker to understand. Beowulf, the oldest surviving English work, cannot be read in its original form without extensive training. The other issue with older works is the matter of preservation. Prior to the modern era, all histories and records were documented by monks, so folk anything would have been missed unless picked up by a printing press or wealthy noble.

I would recommend The Literature of the Middle Ages by W.T.H. Jackson to read more about history of storytelling in Europe. It's quite interesting to see how the way stories are told are constantly evolving, but the story itself has remained the same.

Three elements jump to mind that would make the recording of folklore a popular 19th century pursuit. They, are Romanticism, Nationalism, and the birth of a mass media culture that is closer to our own.

First, Romanticism. Sometimes seen as a reaction to the over-analytical spirit of the enlightenment, Romanticism was an attempt to connect with deeper feeling, and less rational elements that had drifted away from elite culture. One aspect of this was the attempt to connect with nature and the land. The idea that folklore and songs were more authentic had some currency. This led to more elite interest in preserving these works.

Closely linked to this instinct is the idea of nationalism. 19th century intellectuals were very interested in defining and celebrating what made their "people" distinct. Folklore and folk songs provided that opportunity.

Third is the beginnings of Mass Media culture in the 19th century. Print existed centuries before, and large numbers of newspapers for at least a century before. A reading public certainly existed before the 19th century. All of these elements, however, were given tremendous fuel by the Industrial Revolution. A broader public (nowhere near as broad as today, but much broader) had a disposable income, and it was now possible to make real money off of cultural production outside the context of patronage. This had a democratizing effect on what was produced, and increased the volume of pretty much everything. C.A. Bayley's The Birth of the Modern World notes how 19th century industrialization changed and expanded every aspect of intellectual life, from science to organized religion. A growing market for and interest in folklore and folk songs is one aspect of this.

I'll answer your questions with two responses (and if I'm missing the points of your questions, ask for additional discussion). First the question of folklore collecting and motivations. A few quick sources: Richard Dorson, “The Question of Folklore in a New Nation,” Journal of the Folklore Institute 3:3 (December 1966). And see the special issue of Journal of the Folklore Institute 12:2/3 (August-December 1975), which includes the following: Richard Dorson, “National Characteristics of Japanese Folktales”; William A. Wilson, “The Kalevala and Finish Politics”; Abu Saeed Zahurul Hague, “The Use of Folklore in Nationalist Movements and Liberation Struggles: A Case Study of Bangladesh”; Sandra Eminov, “Folklore and Nationalism in Modern China”; and Ilhan Basgov, “Folklore Studies and Nationalism in Turkey.”

I believe the following discussion is relevant to your question and is an excerpt from my article, Ronald M. James, “Cornish Folklore: Context and Opportunity,” Cornish Studies 18, ed. by Philip Payton (Exeter, UK: University of Exeter, 2011):

The professional collection of folklore west of the Tamar began in the nineteenth century a few decades after the Brothers Grimm published material collected in the German states. Jacob and Wilhelm Grimm and those who followed drew inspiration for gathering oral traditions from the fact that various factors were changing society. Rural, agriculturally-based western Europe was transforming into a landscape filled with larger urban centers, with increasing numbers of people earning wages in factories. The old ways were fading.

To this day, much of what motivates collectors is the idea that they must salvage oral tradition before it disappears. Because folklore changes constantly, it often seems the past is slipping away and that if these remnants are not saved, something valuable will be lost forever. During the nineteenth century, circumstances heightened the sense that modernization was devastating a valuable inheritance because change before industrialization was clearly slower and people regarded folk culture as preserving remnants of an ancient time. While it is reasonable to observe that traditions evolve constantly and are not, in fact, static, there is also ample evidence that folklore actually does preserve evidence of former lifeways and beliefs. Early publications represent, therefore, valuable sources from which to learn about the past.

Modernization was not the only factor that motivated early students of oral tradition. Historians occasionally assert that nationalism inspired the early growth of folklore. Today’s folklorists frequently maintain their predecessors were important participants in the struggles for ethnic sovereignty, although testing this assertion remains problematic. Documentation regarding the views of collectors toward nationalism differs from case to case. Unfortunately, information about how early Cornish folklorists regarded the subject is circumstantial at best, but it is possible to place them in a context.

Although the French collector, Charles Perrault (1628-1703), worked in the seventeenth century, Jacob and Wilhelm Grimm are credited with founding the modern discipline of folklore. Jacob (1785-1863), in particular, applied a rigorous scientific methodology to the study of linguistics and oral tradition. Wilhelm (1786-1859), on the other hand, was more interested in how folktales could inspire a body of German literature to advance a national ethos.

The Brothers Grimm promoted German culture, but they did not invent the idea of linking the study of oral tradition with nationalism. Much of this connection can be traced to Johann Gottfried Herder (1744-1803). In his 1784 publication, Ideas on the Philosophy of the History of Mankind, Herder called on Germans and others to use language and popular traditions to inspire a national consciousness. He saw something unique in the essence of the folk, or to use Herder’s language, the Volkgeist. It is no accident that William Thoms, who invented the English word “folklore” in 1846, arrived at a term much like Herder’s. With the assertion that a nation’s folklore was important, Herder broke with Enlightenment thinkers who stressed the universality of humanity rather than regional elements with the potential to divide.

The next generation to answer Herder’s call included the poets Clemens Brentano (1778-1842) and Ludwig Achim von Arnim (1781-1831), as well as the historian Frederick Karl von Savigny (1779-1861). Brentano and von Arnim drew on popular traditions for inspiration, but these early poets were removed from the modern notion of professional folklore collection. Instead, they saw little reason to remain true to their sources: for them, the most important goal was to create a German vernacular literature, based loosely on folk traditions in order to foster national awareness and to inspire a generation of patriots. Together they published a body of poetry titled Des Knaben Wunderhorn—The Boy with the Wonderhorn—between 1805 and 1808. The two hoped to draw attention to the literary potential of German language and culture with what they called Kunstmärchen, which can be translated as “art folktales.” Arnim, in particular, went on to work with the genre, developing it as a distinct form of literature.

This literary tradition had a profound influence on the young Grimm brothers. In fact, they lent an early manuscript of their collection of Märchen (that is, pure folktales) to Brentano with the hope of collaborating. Although it yielded nothing, it was within this tradition that Jacob and Wilhelm Grimm later published their Kinder und Haus Märchen, literally, Children and House Folktales. The Grimm brothers, however, were not influenced solely by a literary tradition. Frederick von Savigny served as a mentor to the young Grimm brothers, stressing the importance of precise historical method to arrive at a better understanding of German heritage. Although Jacob and Wilhelm Grimm were influenced by the sweeping romanticism of Herder, they were not content with exploring popular motifs for literature. Following von Savigny’s inspiring standard for historical scholarship, they invented a process of collecting oral tradition in a manner that approached modern professional standards.

For the second part of your question dealing with what stories sailors might have told in the 17th and 18th centuries, I will draw on research I have been conducting into the maritime folklore of Cornwall. What I have found is that the Cornish frequently took oral tradition common throughout the Atlantic seaboard and adapted stories that were told elsewhere as land-based, transforming them to their maritime environment.

An example of this is provided by the Lenore story made famous by the Gottfried August Bürger with his poem published in 1774. Folklorists refer to this as a folktale, tale type 365, a story told throughout western Europe. It features a dead soldier who returns for his betrothed, threatening to take her to his grave. In Cornwall, he becomes a sailor who seeks to take his bride to a watery grave. And with this, we have a hint of how sailors may have taken material common to the land-based community, but adapted it to their maritime environment. See my article, Ronald M. James, “‘The Spectral Bridegroom’: A Study in Cornish Folklore,” Cornish Studies 20, ed. by Philip Payton (Exeter, UK: University of Exeter, 2013).

Keep in mind that your sailors in question would have been familiar with the distinct genres of the folktales (lengthy fictional accounts, the novels of the folk that usually ended happily) and legends (shorter accounts told to be believed, usually ending horribly and told as a means to offer guidance to avoid the supernatural and to behave properly when avoidance wasn't possible).

An example of legendary material common to our sailors would be the traditions surrounding the mermaid. Depending on where our sailors originated, these creatures would either been half fish/half human or clothed in seal skins that they could shed to appear fully human. My Cornish sailors believed in the former, while the idea of seal folk was the belief found in Scotland, the Shetlands, and Northern Ireland. Because stories about the sea folk are ancient, we can assume that our sailors of your period told them, and this gives us something of an insight about what the folklore might have been like on shipboard. Of course, the maritime world, with its dangers and sometimes unnatural environment, was the perfect setting to spawn all sorts of folk beliefs and traditions. Everything from avoiding whistling on board, avoiding having women on board (especially if they had red hair!) were part of commonly-held traditions for sailors.

For my article on mermaids see (when it is published next year!), “Curses, Vengeance, and Fishtails: The Cornish Mermaid in Perspective,” submitted for acceptance in Cornish Studies 22, ed. by Garry Tregidga (Exeter, UK: University of Exeter, 2015). I may have missed the point of your questions, so please feel free to follow up with additional questions.