Regarding whether they 'forecasted' the fall, check out this perspective. I always find it good to read the official view - and the CIA's public publications actullay have a neutral tone that doesn't sound like 'spin' (of course they would :D).

".. assertions that CIA got it blatantly wrong are unfounded--that charges that CIA did not see and report the economic decline, societal deterioration, and political destabilization that ultimately resulted in the breakup of the Soviet Union are contradicted by the record. Arguments about who was "how right" are of less use, much as we might wish to engage in them.... Certainly there were divergent views and predictions in the CIA--as well as in other parts of the Intelligence Community and in policy agencies and nongovernmental circles--on the potential impact that the economic and societal problems might have on political continuity in the USSR and on the military threat. But there was no disagreement within CIA's Office of Soviet Analysis (SOVA) as to the fact of those enormous problems." https://www.cia.gov/library/center-for-the-study-of-intelligence/csi-publications/csi-studies/studies/97unclass/soviet.html.

My own take is that if even Gorbachev didn't foresee it, so how could the West? Gorbchev didn't intend to collapse the USSR, only save it by reform. When he decided not to use force in eastern europe and east germany the guarantee of violence that kept the system in place was gone. If he had done the what Deng and China did, who knows...