That is the purpose of the minaret. The muzzein went to the top to sing the adhan.

The worshipers were listening to hear the call to prayer. It's always at the same times relative to the positions of the sun, so when it was almost prayer time they were expecting it.

One factor that helped is that things were usually quieter in the past. There were no automobiles or airplanes or air conditioning units making noise. The sound didn't have to carry for miles and miles, and the typical architecture did not include double glazed windows, so you were more likely to hear from inside a building than you might today.

Another factor is that the target audience was within walking distance. It didn't have to go for miles. In more pedestrian times, if you miss hearing it entirely but see lots of people walking toward the masjid, that's your cue. If you arrive after the prayer is finished you can still do it solo.

The Adhan was, and is, called from the Minaret which is the high part of the mosque. As you say, now most Mosques use a PA system but in historic times it was called from the high point so the Muezzin's voice would project further. The five daily prayers are at set times every day, so an observant Muslim would already be preparing before the call went out.