The light would have traveled much faster than the blast, so you would see that before hearing anything. Hearing it would have been coincident with having the blast wave hit.

The bombs don't make the noise you think they do, though. All of the "booms" you hear on most nuclear test footages are just spliced in generic explosion noises. The actual noises are more like a single loud clap followed by echoes. If the clap arrived at the same time you were thrown across the room and your ceiling fell in on you, I imagine you'd probably not remember it quite as distinctively, or be able to separate it from all of the other noise that was involved.

Most of the Hiroshima accounts I have read involve people getting focused on the local damage and thinking that a regular-sized bomb hit their house, rather than one big bomb hitting the whole city. I would guess that some of this is what you expect to see/hear as well — when you hear horse hooves you don't expect to see a zebra.

I've posted one of the very rare instances of live audio at a nuclear test explosion here. It is from some distance away from the blast. It matches up with accounts from the Trinity test.

A bit late, but oh wells.

"Being thrown across the room" actually is the noise from the explosion. An explosion (nuclear or otherwise) transmits energy to its environment through a shockwave traveling through the air (or other medium). That shockwave is a sound wave. It's a sound wave with an odd combination of parameters that don't arise except in the context of an explosion, but nonetheless it is a sound wave.

Here comes the science! A nuclear explosion releases radiation. This radiation is absorbed by the surrounding air, heating it up. It heats up to several thousand degrees in a few milliseconds. The air tries to expand, and increases in pressure. This sudden increase in pressure is what causes the shockwave. The hot air then rises, causing a decrease in pressure about a second later, which creates a sucking effect.

Sound is a wave--air particles moving back and forth, in and out. With the shockwave from an explosion, there is a single out movement and a single in movement. While normal humans can hear sounds in the frequency range of 20 to 20,000 cycles per second, the explosive shockwave consists of only a single cycle. To your point, it has an outrageous amplitude (volume/power), but it doesn't really have a frequency. It exists as a single point, rather than a continuous phenomenon like 'normal' sound.

They would have heard a loud noise as well as seen the flash of light, but light travels faster than sound.

The Japanese expression for the atomic blasts became pika-don, rough literal translation being 'blinding flash-loud roar'. [E. Takemae, Inside GHQ, p.42]