Menander, the ancient Greek playwright, wrote over a hundred comedies and was one of the most popular writers of antiquity, according to our sources. The apostle Paul quoted him in the New Testament, Plutarch praised his works, and where Julius Caesar crossed the Rubicon, he supposedly said, "Let the die be cast!", a line from one of Menander's plays.
Yet his works mostly didn't make it to the present day. Up until the 1900s, all we had of his plays were fragments quoted by other authors. During the last hundred years, matters have improved somewhat since we found the Bodmer Papyri and a couple of other fragments. Yet his only complete play is Dyskolos, and Sami (the second most complete play) still has over a hundred lines missing.
Why didn't more of Menander's works survive? Did they not make the transition from scrolls to codices, or was his work considered less important than that of other playwrights like Euripides or Aristophanes? Or is there some other reason?
The survival of ancient Athenian dramas depended on a variety of factors. There are several things Menander could have fallen foul of. Though he got into some school curricula in the Roman and Byzantine eras, he wasn't in the school canon consistently; he was just a couple of decades too late for Roman-era and Byzantine grammarians to consider him a good teacher of the classic Attic style; he never had scholarly commentaries written on his plays (unlike Aristophanes).
If any of these had gone differently, we'd probably have mediaeval manuscripts of Menander. As it is, the latest evidence of Menander being read -- and, as it happens, taught -- is (I believe) from a 6th century CE school in Gaza.
After thinking about it a bit, I think it comes down to: he was just a little bit too late. Not his own fault, and not reasonable. But there it is. Hellenistic scholars, especially Aristophanes of Byzantium, greatly admired him; Aristophanes famously thought that Homer and Menander were the all-time literary greats. (That sentiment actually outlived the plays themselves: Ausonius repeats it in the 4th century CE, probably without having actually read Menander.) But for 3rd century BCE scholars, Menander was within living memory: Aristophanes' teacher Zenodotos was a contemporary of Menander's. And also by Menander's time the pronunciation of Attic Greek had already evolved into something more resembling koinē. So they didn't make a standard edition, or any commentaries.
Just to be extra clear: for Hellenistic scholars, there were no challenges in the language (because it was all contemporary), no alienness about the social mores depicted (again, contemporary), and no references to specific historical events. Because of that, there was no need for scholarly glosses. So I'd assign a large part of the reason for his loss to the fact that 3rd century BCE scholars thought he was too easy. (Edit. You mentioned Plutarch's praise of Menander: there's another passage, in his Convivial Questions, where he mentions that when anyone recites Old Comedy at a party, all the guests practically have to have a grammarian at their elbow to explain the archaic vocabulary and the contemporary references. Menander didn't require that kind of expertise.)
That, plus the fact that Roman- and Byzantine-era grammarians were terrible snobs about the Attic dialect. If Menander's career had started just a little bit before Alexander, he'd have stood a good chance of being lumped in among the Atticists, alongside Demosthenes. But he was on the wrong side of that watershed, and for grammarians of the Second Sophistic (roughly 100-300 CE), that made him less of a paradigm to follow. That, I suspect, is when Menander started to be actually lost (that's also about the time of the transition from scrolls to codices, a factor that you pointed out in your question). But that loss could have been averted if Aristophanes' decision that there was no need for a commentary or a standard edition had gone the other way.
There's always a multitude of reasons for the non-survival of any text, and the biggest one is always that every time a scribe didn't copy Menander, that brought Menander one step closer to extinction. But I think a more boiled-down version would look something like the above.
where Julius Caesar crossed the Rubicon, he supposedly said, "Let the die be cast!", a line from one of Menander's plays.
Is it known what the original context for this line was in the actual play?
Because it's certainly fascinating to know that he took it from a play. Like how a public figure nowadays would be quoting from a movie or something.