So first, yes, many people didn't travel far, but a lot of recent work has shown that medieval horizons were much broader than people first thought. Pulling a wife from a neighboring village would not be like venturing off to find the grail.

Second, villages weren't so very small, by and large - I don't know of an actual study of average size, but 50 wouldn't be too absurd.

Third, and most important, the Church forbade marriage within seven degrees of consanguinity. While there were often dispensations given to nobility (see the aformentioned Hapsburgs), the process would be cost-prohibitive for almost everyone else. On the other hand, few people - even noble families - could trace their lineages back that far, so the extreme limits of the rule were probably not well enforced.

Incest is not a correct term for the phenomenon, a better term is inbreeding. Incest involves only close family members, and such marriages generally didn't happen (cousin marriages did, but that's not incest).

Inbreeding within a village isn't a serious threat, but if the village was small and it continued over many generation, it could pose a problem. That is why people in different villages often wore different costumes and people married their daughters off to neighbouring villages.