They had satellite imagery of the groom lake facility, aka "area 51", and so could make some good guesses about its purpose. It features an incredibly long runway which then runs right into a salt flat (serving as extra emergency runway). That would have been a rather certain indicator of research and testing in experimental and especially high speed aircraft.

Note also that despite a great deal of attempted secrecy, programs like the U2 and SR-71/A-12 were not secret for long. LBJ revealed the SR-71 program only 2 years after the first A-12 flight. The U2 was tracked by Soviet radar within the first year of operation. So they had an idea of the sorts of planes the US was building in secret and a good idea of where some of that was happening.

They almost certainly were very much less aware of the development of stealth aircraft, Have Blue and the F117, however, which used groom lake facilities extensively during development.