I can understand how we might decipher the hieroglyphs, but how can we possibly know the sounds (e.g., names like Thoth or Sutekh) when we have no record of anyone speaking the language? Surely we can't infer it just from the hieroglyphs?
The short answer is: "We don't."
The slightly longer answer is that we know (mostly) how Coptic sounded, because the Coptic alphabet records the vowels and the language is still used as the liturgical language of the Coptic Orthodox Church, so we often use Coptic vocalizations for common things. For instance, the name "Thoth" survives as the name of the first month of the year in the Coptic calendar as ⲑⲱⲟⲩⲧ/ⲑⲟⲟⲩⲧ = Thowt, and in Greek texts as Θωθ = Thoth. "Thoth" is good enough for most purposes, it's similar enough in Coptic, no one overthinks it.
Going beyond the simplistic answer of just using the Coptic or Greek versions, we can infer some things. We know that the name of the god Seth survives as Ⲥⲏⲑ = Seth, and we have Late Egyptian spellings like 𓋴𓏲𓏏𓄡𓁣 = swtẖ = ['sutɛx] (in Egyptological pronunciation). From various sources of evidence—especially Egyptian words in foreign scripts—we know that many [u] sounds became Coptic <ⲏ>,^1 so we can infer from the hieroglyphic semivowel character <𓏲> and later Coptic <ⲏ> that the first sound of this word was [u]. Then because the Egyptian word has three root consonants: 𓋴, 𓏏, 𓄡, and because we generally believe that Coptic omits or elides short vowel sounds,^2 we can speculate that there was a second [ɛ] (or similar) that was subsequently lost. Putting all of that together gives: Late Egyptian ['sutɛç] → Coptic ['setʰ]. It's not a bad guess given all the evidence.
And that's mainly how it's done, by considering what evidence we have and using reasonable speculation to bring it all together into a coherent whole. That works well for some things, not at all for others. In particular, grammatical forms that do not survive into Coptic are much harder to reconstruct. The dominant Egyptian verb form for most of the language's history, the so-called sḏm.f, does not survive into Coptic well at all. That we guess about in the same way but with less certainty.
If anyone is interested in directly learning all these sources of evidence so that you can do this on the fly like I just did, you might sign up for the free online course I'm teaching on exactly this subject. The first class was last Saturday, so you can still catch up.
The short answer is that we don't, not really. Ancient Egyptian pronunciation is a multi-source reconstruction, not a positive assertion about how the real people actually spoke. As an aside, it's important to remember that "ancient Egyptian" wasn't a single, unified language. Its recorded history spans more than 2000 years — consider the differences between modern English and the "English" of 2000 years ago — and like every other major language, it had wide dialect variation.
Most of the common Egyptian names we know today — people, deities and cities — are actually transliterations that come to us via the Greeks, usually established during the Ptolemaic period (305 BCE - 30 CE). Reconstructed native pronunciations of those names are often quite different. For instance, Thoth was a Greek reflex of a name more accurately rendered — transliterated directly from the hieroglyphs — as ḏḥwty, with a reconstructed pronunciation of something in the neighborhood of *[t͡ʃʼi.ˈħau.tʰiː] or **[ci.ˈħau.tʰiː]*. While the values of the consonants are fairly well established, the values of the vowels are still pretty mysterious. In general, though, we don't know the exact phonetics of ancient Egyptian and likely never will.
As for how we arrived at these pronunciations, I'm absolutely thrilled to be able to introduce you to the field of comparative historical linguistics. This is a field of academic research concerned with analyzing the genetic and historical relationships between languages. Its most important products are the reconstructions of ancient or prehistorical proto-languages, the hypothetical ancestors of modern language families, but it's also capable of filling in some of the blanks of languages (like ancient Egyptian) that are attested in written but not spoken form.
As for ancient Egyptian in particular, there are three primary avenues for reconstructing its phonetics:
The details of these methodologies are a book-length topic, at minimum, but man, are they interesting. For a glance at the development (both reconstructed and attested) for a handful of words, take a look at the chart at https://en.wiktionary.org/wiki/Appendix:Egyptian_pronunciation.
This topic has been studied for almost two centuries at this point, but there's some very good recent scholarship. I would point you in particular to the work of James P. Allen, an Egyptologist at Brown University specializing in Middle Egyptian.