There was a very similar question recently, but I can't find it to link and I've written about similar topics before. I did find this answer from u/Bentresh on decipherment in general, specifically using the undeciphered Indus Script as an focus. If that specific topic catches your interest, I wrote an answer about some new developments more recently.

There are essentially two basic paths to deciphering forgotten scripts. One is using multilingual inscriptions or documents with the same message in different languages and working from a known language to identify words in the unknown ones, as with the Rosetta Stone (and actually a number of other artifacts used as cross references). The other is to work backward from a known language.

Both require guesswork to identify what type of script you're looking at. Is it an alphabet, an abjad, a syllabary, logographic, etc. That isn't a total shot in the dark because different types of scripts usually have more or fewer characters than the others. If every word has its own symbol (logograms), more characters are needed than if each phoneme has a letter (alphabets), for example. Working backward from a known language requires the added step of correctly guessing what language, or languages, the forgotten script is connected to. In either case, it is immensely helpful for the undeciphered text to mention known proper nouns because they are less susceptible to change over time than commonly used words.

As an example from my own field, Old Persian cuneiform was deciphered by working back from modern Persian, Middle Persian, Avestan (the language of Zoroastrian scripture), and Sanskrit. As the longest known Old Persian text, the Behistun Inscription of Darius the Great was the primary text used in the process. With the additional knowledge of common ancient Persian names and historical events provided by Classical Greek and Latin sources, European scholars made some educated guesses.

The style of the monument and use of cuneiform rather than Middle Persian Pahlavi script indicated that it was quite old, and the oldest and greatest empire they knew of from the region was the Achaemenid Persian Empire. Their legacy was well documented and at least some of it remained present in later forms of the Persian language, so they guessed that one of the three scripts on the monument was a form of Persian and that some of the repeated words were the names of known kings or gods. As the language of the ruling elite, they guessed that Old Persian was the language used for the most central and best organized version of the monument's text. They were lucky, and that guess was correct, because the Old Persian translation was also the easiest to reach and transcribe.

Knowledge of later Persian provided what words they should expect to see, and the more archaic examples Avestan and Sanskrit provided a basis for the probable differences in grammar, syntax, and pronunciation. They were also lucky that Old Persian is an alphabetic script, meaning they had fewer characters to guess their way through to identify likely analogs for proper names. From there, it was just a matter of identifying which names they recognized: Darius = Darayavaush, Hystaspes = Vishtaspa, Ahura Mazda = Auramazda, and so on. With enough properly identified characters, they moved onto words that contained all or mostly known characters and matching them with known words from the established languages, filling in gaps with logical assumptions based on the available evidence. If a guess was wrong, it quickly became apparent once whole phrases could be translated because incorrect guesses did not make sense in context.

Once Behistun's Old Persian was deciphered, it actually served as the "Rosetta Stone" for Akkadian and Elamite as well.