There were those in Hiroshima who later reported that they worried that a big attack would be coming, since they had not been attacked and so many other cities had been. It is hard to determine whether that is just confirmation bias. I've seen nothing that indicates the Japanese public or military were worried about an atomic bomb attack, though — this was just not in their list of possible outcomes (even those that knew of the idea of nuclear weapons did not consider them a short-term threat). They were totally ignorant of US progress in this area.