My understanding is that throughout the Heian era of Japanese history, the establishment of shōen (untaxed estates) slowly but steadily increased, bleeding the imperial economic power base dry in favor of various local lords and temples. These could in turn afford to maintain large private armies and significant influence on the imperial court. At some point the Fujiwara are practically in full control of imperial succession, until the Hōgen rebellion and subsequent Genpei war puts a stop to that and cements Minamoto no Yoritomo and the Kamakura shogunate as the de facto ruling political power for the next several centuries.
In short, it seems that the shōen system was a significant contributing factor to the decline of the imperial court as the dominant political power in classical Japan and the ushering in of the shogunate eras. What I'm wondering is, how could the imperial aristocracy allow the shōen system to continue for so long? Were there attempts at re-taxating or reappropriating estates? Was the possibility of this kind of catastrophic loss of control debated at the court at all? In short, did no one see this coming?
You're correct that the spread of shoen was a key factor in the weakening of the imperial government's authority in the provinces and the development of a new warrior class. But one factor I think you may be overlooking (as your post mentions "local lords and temples" as being the benificiaries of the shoen) is that the shoen were also incredibly beneficial to the nobility in the capital as well, who had tremendous influence over the government. In addition to establishing their own shoen, they were frequently given rights to other shoen in exchange for using their influence to ensure immunity from taxation. So the aristocracy had a lot to gain from protecting the shoen. This may seem short-sighted, but the shoen developed over the course of centuries so they certainly weren't an immediate threat.
That said, the central government did attempt to crack down on the growth of shoen from the late ninth century on, although this was generally more of an attempt to curb abuses than to eliminate growth altogether (no one ever wanted to get rid of them entirely, and even emperors created shoen to benefit relatives). These efforts experienced limited success due to interference by the nobility and a perhaps excessive desire to show deference to past laws and practices. One particularly famous attempt was a 1069 law by Emperor Go-Sanjo which abolished all shoen established after 1045 and created a government office (the Kiroku Shoen Kenkeijo) to examine the paperwork for earlier shoen to check their legitimacy. By the late Heian period, there was a constant struggle over taxation between provincial officials on the one hand and shoen officials and their noble backers on the other.
It's also important to remember that while the shoen allowed the nobles to become increasingly economically independent of the government, they still represented a minority of land during the Heian period; an estimated sixty to seventy percent of land was still under the control of provincial officials. They weren't a critical threat to the imperial finances.
Sources:
Cornelius J. Kiley, "Estate and Property in the Late Heian Period," Medieval Japan: Essays in Institutional History
Cornelius J. Kiley, "Provincial Administration and Land Tenure in Early Heian," Cambridge History of Japan, Vol. 2