Short answer: Yes, much of it would be considered tasty.

Long answer: Medieval cuisine was rich and varied, and it obviously differed greatly from place to place. I'm going to focus on the cuisine of medieval England, because I recently read an excellent book on the subject.

To begin with, most medieval Englanders would not have eaten three meals as we do today, but just two. They also would not have had access to any New World crops such as corn, potatoes, tomatoes, or avocados, or animals such as turkeys. Their options would have been limited in other ways--unless you were very wealthy, you mostly would have eaten local food grown nearby. Additionally, the Catholic Church forbade eating meat on Wednesday, Friday, and Saturday, and during the seasons of Lent and Advent (all this amounts to about half the year). Finally, the medieval diet was based on seasonal availability, so fresh fruit would have been mostly seen during autumn, and vegetables would have been seasonable as well.

That said, medieval cuisine could be quite elaborate and delicious (at least for aristocrats and wealthy merchants). Peasants and ordinary folk would eat a lot of bread, pottage (which could be made with anything from leeks to oats to peas), vegetables, cheese, and beer or ale, with meat served at feast days and special occasions. But lords and ladies would have very complicated, multi-course meals, centered around meat since only well-off people could afford to have it every day.

The first course of a lord's meal would often include boiled or baked meats in sauces. A common dish was Leche Lombard, which was pork, eggs, pepper, cloves, currants, dates, and sugar all boiled in a bladder, sliced, and served with sauce. Fruits and vegetables would have been cooked since many people thought they caused disease if they were served raw. Fruit dishes included apples and pears with sugar or syrup and baked quinces. Dishes would be flavored with spices like nutmeg, pepper, and ginger.

Meats were delicately carved, and there were many more of them then people commonly eat today. Beef, pork, and chicken were common, but people also ate rabbits, roast venison, pigeons, larks, swallows, and other kinds of birds.

On days when meat was prohibited, the focus would shift to fish, which was highly prized. Fishes eaten included bream, pike, herring, lampreys, eels, trout, halibut, sea bass, and many others. Common dishes included pike in a sauce made with cinnamon, ginger, salt, breadcrumbs, and stock, or fishes poached in various ways. Nobles and kings especially prized sturgeon and salmon. More unusual foods included seals, porpoises, dolphins, puffins, beavers, and even whales. The prohibition on eating meat only applied to land animals, not creatures that lived in the water.

So yes, much of medieval cuisine would please modern-day taste buds. It would certainly be different and probably require some getting used to, but cooks back then knew how to make delicious food. And yes, people ate a wider variety of animals then we do today.

Sources: "The Time Traveler's Guide to Medieval England" by Ian Mortimer.

http://www.foodtimeline.org/foodfaq3.html#medieval

And here is a cookbook from 1390 A.D., called "The Forme of Cury", which has 205 recipes and is written in Middle English: http://www.pbm.com/~lindahl/foc/