In the case of ancient Egyptian, we know how to pronounce it because of Champollion's work on the Rosetta Stone where he matched hieroglyphs with the greek letters of names - like Ptolemy, Cleopatra and Alexander - and discovered that Egyptian hieroglyphs are phonetic and not symbolic. There's one vogal that's missing in Ancient Egyptian "alphabet": E; so when you make the transliteration of hieroglyphs you put "e" between two consonants to facilitate the pronunciation. It's important to note that this doesn't mean we know how ancient egyptians spoke three thousand years ago.

Students of Latin often ask this question: "How do we know that Latin was pronounced a certain way, and not another way?"

The answer is that we have strong evidence from multiple sources for pronunciation. First and foremost, late antique grammarians directly discuss points of pronunciation, both in their own day and looking back into (their) past. They were often concerned with peculiarities of the language.

Second, we have the transliteration of other words in Latin from other languages. How the Romans spelled Greek, Egyptian, Persian words tells us a lot about how Latin words were formed in the mouth. Names, in particular, are very useful, and from inscriptions we have thousands of Latin transliterations of Greek and Celtic names (for instance).

Speaking of inscriptions: Latin writing on stone provides a permanent time capsule of orthographic representation of pronunciation, and allows us to see how the language changed through time. A good example is the duenos inscription.

Phonological comparison with other languages in the same group is also useful. Latin is an Indo-European language, and shares a certain phonological legacy with all sorts of different "sibling" languages (like Celtic, foremost, but also more distantly Greek, Germanic, etc). Phonological outcomes from Proto-Indo-European, especially across all the descendant languages, provides a mountain of evidence for possible pronunciations.