My perspective on this is a little at odds with traditional doctrine; as such, I fully expect the mods to remove it as speculation, but I’m willing to risk my flair over it to set the record straight. Basically, the Sith “Rule of Two” is a metaphorical statement on Sith doctrine and power theory: that the master should constantly be challenged and replaced by the student. It is not an ironclad rule, and history reflects as much.
If the “Rule of Two” were strictly enforced, formal Sith training would be a contradiction in terms: how could more than one student ever be trained, and how could Sith armies ever hope to field Force-users in strength? They could not, and in fact, when push comes to shove, we see the Sith willing to bend their rules for practical effect (as you might expect of a Sith).
For example, at various points in history there has been a thriving Sith academy on Korriban, alternately visited and supported by no less than Revan himself. And, during the most recent Galactic Civil War, Emperor Palpatine, as reigning “dark lord,” employed or benefited from the counsel and support of a number of Sith apprentices, including Darth Maul and (ultimately) Lord Vader.
What Master Yoda was likely referring to was not the theory of the Rule of Two, but how it was carried out in practice, where, regardless of how many lower-ranked Force users the Sith might employ, the Sith heirarchy comes to be dominated by a two-person master/apprentice team. Even if Sith were believed absent from the galactic community at outset of the Galactic Civil War, remember, Yoda was nearly a millennium old, and would have been well-schooled in Jedi and Sith history during his time with the Old Republic’s Council. There, many historical examples -- Revan and Malak come to mind -- would have presented themselves for Yoda to study, from which he could infer the likely existence of a similar dyad.