I read that they would retire to increase their power relative to the Fujiwara clan, but I wasn't clear how that worked.
Well, Gosanjō’s abdication and his installation of Shirawaka as the new Tenno was more or less a choice of opportunity: at the time, he was not under control of a Fujiwara gaiseki (maternal father/grandfather), since his mother was an imperial princess. This enabled him to disregard the Fujiwara more than his predecessors could. He also likely did not want Shirakawa to be his main successor, but rather Shirakawa’s brother Sanehito-shinnō, who he declared crown prince upon his abdication in 1073; however, Gosanjō died only five months after his abdication, so its not clear what he had envisioned for the future.
Now, Shirakawa himself was blessed with the luck that the most powerful members of the Fujiwara regent family who had wielded a lot of influence within the imperial court—Fujiwara no Yorimichi, the late Emperor Ichijō’s wife Fujiwara no Shōshi, and her brother Norimichi—all died in the years 1074/75. His brother, the crown prince, also passed away in 1085, enabling him to choose his successor himself.
Shirakawa was also gifted with one thing not many of his predecessors had: longevity. He was able to use his authority as reigning emperor to determine his own successor (Horikawa), but also place his grandson (Toba), and even great-grandson (Sutoku) on the throne. He could do so by relying on his personal authority as patriarch of the imperial family. This was new, since before him, the head of the imperial family de facto was the reigning emperor; the concept of emperor and head of the imperial family became separated. (Incidentally, this apparently also led to changing perceptions of who, or what, the emperor was, seeing him increasingly as a mere “human,” as can be gleamed from accounts in medieval diaries, chronicles, and fiction)
Furthermore, if we consider that, at the time, the imperial government was very entrenched in traditions, precedent, and a bureaucracy defined by, for all intents and purposes inheritable offices—in other words, fixated by institutions of both the administration as well as social kind—and that this was mostly dominated by the Fujiwara, the establishment of a new center of power outside of this traditional framework enabled the imperial family to react to the changing times, where local society was experiencing rapid reorganization by the intrusion of the ever-expanding estate system.
Especially this estate system became increasingly important for the court elites as a means of income: the emperor himself, by virtue of, in theory, owning all land anyway, could not participate in the fun game of amassing estates—something that the abdicated emperor very much could, since he was “merely” the head of the imperial family, free from all the institutionalized ritual and regulations.
As head of the imperial family, Shirakawa was able to establish a household administration of his own; in comparison, the imperial government itself is de facto the emperor’s “household administration,” since it had absorbed this very same organ centuries earlier. Accordingly, he was free to choose his own advisers, officials, and confidants, able to disregard the precedents and hierarchies of the established aristocratic bureaucracy, and thus also to distance himself from the (Fujiwara) elites, who occupied most of the top spots within this hierarchy. The latter was probably not part of some grand design, but rather a result; in general, insei rule facilitated personal factors such as kinship ties, but also of friendship, sympathy, and patronage to play a much larger role for attaining influence within the court than before.
Either way, for as long as the Tenno himself was a minor, the abdicated emperor, father of the emperor, could thus become the factual central authority that was able to exercise his own personal will (although, of course, always in cooperation with the influential court elites) by virtue of being father to the emperor. This arrangement only became more complicated if the reigning emperor became an adult—since this, now, would establish a second center of authority with its own will, and the legitimacy to exercise that will, which therefore would require either cooperation within the imperial family or potentially lead to conflict (the power struggle between Shirakawa and Toba is quite exemplary of the latter).
According to Hongō Keiko, the emergence of insei government was therefore not necessarily the result of a conscious plan, but rather a consequence of the combination of several factors: a temporary crisis within the Fujiwara elite which led to a power vacuum, Shirakawa’s longevity permitting him to decide the imperial succession, and also wider currents of societal change which reorganized the provincial government, and the resulting reorganization of politics. All of this would also be further influenced by the political struggles of the twelfth century which placed increasing significance on professional warrior elites within the arena of politics.
Main References:
Hongō Keiko, “Inseiron,” in: Iwanami kōza Nihon rekishi, vol. 6: Chūsei 1, Tokyo: Iwanami Shoten, 2013, 29–60.
Satō Shin’ichi, Nihon no chūsei kokka, Tokyo: Iwanami Shoten, 2020 (first published 1983).