They were never the majority group of Shi'ia. You're thinking Zaydis who were the majority and became the minority. Ismailis had a caliphate but the lands they controlled were majority Sunni (and Ithna Ashari)