As it's worth studying, I'll quote the relevant section for those who haven't read Ian Mortimer's The Time Traveler's Guide to Medieval England:

You would have thought that fish would provide a welcome, non-meat source of protein for the peasantry. Welcome it is indeed, but it does not figure greatly in most rural families’ diets. For those living inland, there are obvious problems of obtaining fresh fish – transportation adding greatly to the cost. But there is another, underlying reason. The Church’s prohibition of meat consumption on certain days helps to create a strong demand for fish among the nobility, gentry and clergy. For this reason fish is expensive. Manors where the peasants are allowed to fish for themselves (like Alrewas in Staffordshire), are the exception, not the rule^7 .

That footnote is where Ian Mortimer comes unstuck. The citation there is for page 157 of Christopher Dyer's Standards of Living in the Later Middle Ages which, in discussing the variety of a peasants diet, says:

Fishing provided food for peasant households, as well as saleable catches. At Alrewas (Staffordshire) in the Trent valley, a good number of tenants were allowed, according to customs written down in 1342, to fish on the meatless days (Wednesdays, Fridays and Saturdays) 'for their table'.

As you can see, Dyer's tone suggests that peasants fishing for their supper was a common occurrence, not something that was rarely done. Although he doesn't expand on this in Standards of Living, he has an entire chapter dedicated to the freshwater fish in Everyday Life in Medieval England, a book which Mortimer has certainly read and cites elsewhere in the Guide. In it, Dyer points out that ponds and fishing rights in rivers were increasingly leased out to commoners during the 14th century in exchange for a cash rent and that the vast majority of these fish, some of which cost as little as 1/4d each, were sold on the open market. Additionally, fenland areas saw heavy fishing by the peasantry even before this, as the stock of fish was plentiful.

Another aspect to examine is whether the Church's prohibition on eating freshwater fish drove up the price to the point where the common man couldn't afford to eat them. This is also based on a false premise.

In "‘Dark Age Economics’ revisited: the English fish bone evidence AD 600-1600", James H. Barrett et. al. examined the archaeological evidence in England and compared it to similar studies in Continental Europe, finding that saltwater fish were mostly confined to the coastal areas prior to the tenth century in England. It then rapidly replaced most of the freshwater fish in urban centers during the late tenth to late 11th century and gradually spread out to the rural areas. This is a pattern repeated elsewhere in the North Sea region, although inland France only sees saltwater fish replacing freshwater fish in the early 13th century.

The cause of this is, in their opinion, largely a shift in economic conditions that the trade also helped grow. Ships rapidly increased in tonnage at the start of the 11th century, allowing for better carriage of bulk low value goods, while the growth of urban centers provided a ready market. This in turn created an incentive for the bulk production of dried or salted sea fish for both export and transport inland to the growing towns.

Another combination of factors also likely contributed to the increased prominence of saltwater fish in the medieval diet. While Barrett et. al. don't believe the assemblages of fish bones they studied can be used to demonstrate that the number of freshwater fish overall declined, as opposed to their consumption declining, the evidence produced by Richard C. Hoffman in "Economic Development and Aquatic Ecosystems in Medieval Europe" is quite compelling.

As European population exploded in the High Middle Ages, more than tripling in some areas, large amounts of land needed to be cleared of forest to make way for crops. This increased peak run off after rains and during snow melts, but decreased flows in other parts of the year, resulting in increased riverbank erosion during peak flows and increased deposition of sediment during lower flows, which can all be seen in core samples from rivers and excavations of silted up areas/dry riverbeds. This, in combination with increased effluement from urban settlements and increased damming of rivers and streams to power water mills, substantially changed the ecology of the river systems from cool, clear, fast moving streams to much warmer, muddied and slower waters that were much less suitable for native fish. At the same time, the increased population increased demand for fish, which tended to result in overfishing unless restrictions were put in place, as they increasingly were from the thirteenth century.

Fishing rights also tended to fall into aristocratic and monastic hands, both through their efforts to secure economic control over as much of the land as they could and from their construction of water mills, so that by 1200 most fishing rights were in their hands. Ironically, this is also when they started first paying men to catch their fish and then permitted the selling of those fishing rights to other men so that, by the end of the thirteenth century, many parts of Europe had dedicated fishermen who merely payed a cash rent and/or portion of their catch in exchange for the rights to sell the fish as they wished. This increased exponentially after the Black Death so that, by the end of the fourteenth century, very few of the clergy or aristocracy maintained exclusive fishing rights.

At the same time, aquaculture was increasing on the Continent, with quite elaborate systems developing to raise carp and eels for more general consumption and pike for more select customers, although this didn't take off quite so much in England. Although even the carp weren't exactly "cheap", especially in comparison to saltwater fish, they were cheaper than freshwater fish had been previously and, if not a regular food item, it was far from a rare one.

The picture created by this is, I suggest, a much more complicated one than Mortimer suggests. As Dyer points out (and Hoffman, citing evidence from Europe and England, supports) is that peasants were often able to fish for their own tables on certain days or during certain periods, even when the fishing rights were still held by the lay or ecclesiastical lord, and that during the 14th century these fishing rights were almost entirely rented out the peasantry in exchange for cash payments. Additionally, the relative expense of freshwater fish was likely not the result of the nobility, gentry and clergy consuming the bulk of the fish, but a combination of environmental, demographic and economic factors that saw decreased stocks of fish as a result of environmental factors, increased overall demand from a larger population and also a perceived increase in prestige as a result of cheaper saltwater fishing being available in abundance.