I read that Beer/Cider/Wine was drunk as water to poor in quality to be drunk in middle ages.

This is frequently found in books (especially general readership books) but this can not be substantiated with primary sources and fails many logic tests.

The main point to consider is that poor water quality is a problem when you have high population density in an area without an adequate sewer system and no infrastructure to bring in fresh water. For starters, in the middle ages there were very few high population density areas in western Europe. Until the 18th century it was a world that was by far almost exclusively rural and agrarian. People built small villages and homesteads in areas where they had easy access to fresh clean water and eventually this was even easier with the advent of artesian wells. Lack of clean water was just not an issue faced by people prior to widespread urbanization.

But even in the biggest cities that did exist, you might be surprised at how sophisticated water delivery systems were in the middle ages. London had the "great conduit" for example. It seems that the poorest of the poor would have had to drink river water but they were a minority in a minority. Clean water was simply not an issue for the average person living in the middle ages.

But by the 19th century water was clearly a large part of the diet again (as seen from the amount of people infected by cholera in water pumps). When did this shift back to beer to water occur?

There was never a shift. 19th century London had the cholera outbreak because population density was finally outstripping their ability to have clean water.