They had paintings of things to indicate their name or what they sold. Or a symbol, like the red and white striped poles that indicated a barber. You'll see today lots of pubs keep up this tradition. Like a classic old pub, The Eagle and Child, that dates from the 17th century in Oxford. It has a painting of an eagle and baby on its sign. Though keep in mind that it used to be that people traveled a lot less, so would likely be familiar with all of the stores around them from a young age.

I can't remember the specific titles, but several chapters in Ancient Literacies deal with this. The basic idea is that viewing literacy as a binary is a mistake--there are all sorts of shades in between, and more importantly, there are many types of literacy. There would be a general sort of literacy among the urban populations, such as the ability to read small signs. In other words, you don't need to be able to read Virgil to understand a sign saying CARNIFAX.