In English, we can read old texts and find out how words used to be pronounced based on their spelling. However, since Chinese uses characters, not an alphabet, is it possible to find out how words in Old Chinese were pronounced?

This is more of a question for r/linguistics, but I can provide a simple answer on the techniques used

Yes (and further, the techniques are the same ones necessary to reconstruct details of Old English pronunciation that were too fine-grained for the spelling system such as the exact qualities of vowels!).

The predominant method in Linguistics is what is called the Comparative Method, which searches for regular systematic correspondences in languages known to be related. For instance, looking at the word for "Time" in romance languages: French Temps, Catalan Temps, Spanish Tiempo, Portugese Tempo, Romanian Timp

We can see these look related. However, just because a language uses the same words doesn't mean it's related. A word could be borrowed, like many words are in English. In order to be related, there needs to be a set of rules that you can use to permute the language back to a predecessor. For instance, Latin lost the endings -m and -s on words, final -us became -o in spanish and dropped entirely by modern french, C in front of E or I became pronounced like S in French or English CH in Italian, et cetera.

These rules are historically real- language (for some reason) changes according to rules that speakers barely notice and certainly don't intentionally follow, such as "in our accent we pronounce k-e and k-i with a y in the middle" (which was a true intermediary stage between "Caesarem" (Ky-zah-rem) becoming Cesare (Chay-zah-reh)). These differences sometimes add up until you get an entirely new language because people can't understand eachother at all. Languages only shift in somewhat predictable ways- A never turns into R for no reason, or Z into M in every second syllable- there are almost always good phonetic reasons why a person might 'slur' or 'mispronounce' a word that way, as language change usually follows the path of least resistance. This will be important in a second.

One hint that two languages are related is sharing common grammatical structures- languages don't usually borrow grammar like they borrow words. This is one way to, for instance, tell that English and French aren't as related as English and German or French and Spanish. It also tells us Japanese is unrelated to Chinese, despite sharing some aspects of the writing system and sharing words.

But there are languages related to Chinese- Chinese is in a family called Sino-Tibetan. It includes, amongst other languages, alll the varieties of Chinese (Mandarin and its siblings such as Cantonese), Tibetan and Burmese, as well as many other smaller languages. By comparing the words we know are native in these languages and tracing back what the original pronunciations were by trying to find a form they both could have evolved from before these languages diverged, we can attempt to derive all the way Proto-Sino-Tibetan. Old Chinese would naturally be found partway back to Proto-Sino-Tibetan, the ancestor of all Chinese forms but after the less related Tibeto-Burmese languages diverged (according to traditional views of the Sino-Tibetan family tree). These derivations are only approximations however, since we can never be fully certain we reconstructed it right (there might be more evidence in extinct languages we are missing, for instance). Utilising other evidence such as inscriptions or dictionaries or loanwords from the right period is vital to getting these reconstructions right.

Borrowings can be useful too! We can use these to look back in time and see the pronunciation of words. Japanese, Korea, and Vietnamese borrowed words from Chinese at numerous periods, preserving archaic pronunciations. For instance, modern Mandarin for 7 "Qi" (pronounced something like "Chyee") was borrowed in Japanese as "Shichi", which in earlier centuries was pronounced "Siti" before Japanese shifted to its modern pronunciation. This is close to the reconstructed middle chinese form (which would be pronounced something like English "Shyeet"). Japanese lacks Old Chinese era borrowings, but I cannot state there are no known borrowings that could help in reconstructing that era, and it's a potential technique.

Chinese also had dictionaries which explained how to pronounce characters, using rhymes- "it starts with the sound in ___ and finishes with the sound in ____" (such as "Ring is like the beginning of Rose plus the end of King"). These can be used to reconstruct some aspects of Chinese pronunciation, in how words rhymed or not, but only for the speaker who wrote it in the time that they did. The oldest ones date back to middle chinese, which is a useful resource for helping to reconstruct old chinese via the comparative method.

Most of these will get you pretty well back into Middle Chinese, but Old Chinese is by its nature more speculative- reconstructions differ more significantly than Middle Chinese, like how reconstructions of Proto-Indo-European (the fundament of almost all European, Persian and northern Indian languages) is more speculative than the reconstruction of Proto-Germanic.