You can probably find a better explanation in /r/linguistics, but essentially what we know about Old and Middle Chinese is based on a combination of the comparative method and the resource of various rime dictionaries from premodern China.

The comparative method is the typical way in which linguists derive protolanguages and common ancestors from existing related languages. Fortunately for Chinese, we have a large corpus of existing information in the form of the many related Chinese languages (Mandarin, Cantonese, Wu, Min) that are still spoken today. In addition, many southern varieties of Chinese are seen as being more conservative in terms of phonology than languages like Mandarin, which gives linguistics more information to work with. Additional information can also be found in countries in the Sinosphere such as Korea, Japan and Vietnam, whose languages borrowed huge amounts of vocabulary from Chinese. Because many of these borrowings took place several hundred years ago, they are valuable in understanding the nature of the Chinese language at that particular point in time.

Rime dictionaries such as the Qieyun were originally derived for the purposes of composing poetry, but are helpful to linguists nonetheless because of their use of the fanqie method to describe the pronunciation of characters. Essentially, each character is described by two other characters: one which has the same initial consonant, the other which has the same final rime. From this, linguistics can begin to extrapolate where convergences and divergences occured in pronunciation, using it as a tool to help guide work using the comparative method.

Because the reconstruction of languages is an inexact science, there is always some degree of uncertainty, but it's not like we turn up empty-handed when searching for resources on Chinese.