Given the cost (both time and money) of hand-copying books prior to the invention of the printing press, would/could there have been bookstores in antiquity, or is the entire concept of 'a store dedicated to selling mainly/only books' a completely modern one?
There were indeed bookstores in antiquity! In Rome, most prominently in the imperial period, we find references to the buying and selling of books from dedicated booksellers. Readership wasn’t an issue, because we do see booksellers and people must have been buying them, although I’ll note that in addition to the massively contentious issue that is the question of literacy in antiquity, we don’t have a ton that we can say with certainty about who patronized bookshops in antiquity because of lack of solid evidence. While costs weren’t so low that a book market entirely replaced traditional circulation methods for new works, they also weren’t so prohibitive as to keep bookselling from being potentially profitable.
One place we tend to see references to the sale of books and booksellers is actually Latin poetry, where both the text’s material status as physical text and the commodification of literary products are important literary topoi, and it’s poems that feature these themes that sometimes mention where to acquire books of poetry. One clear example of this comes out of Martial’s Epigrammata:
You ask me, Quintus, to give you a copy of my books.
I don’t have them, but the bookseller Tryphon does.
“What? Am I supposed to pay real money for nonsense (nugae) and call myself sane while I buy your poetry?
I am not,” you say, “about to do something that stupid.” Well neither am I. (Mart. Ep. 4.72)
The poem’s main focus is on amicitia, friendship, and on the value (social, economic, literary) of poetry, but part of the issue is one of expense. Martial could copy out his books at his own expense and give them to a bad friend, or Quintus can go buy them himself from a store like people Martial doesn’t know personally. This last point gives an indication that there may have still been, among elite readers active in literary circles, a social value to receiving a copy from the author themself. While Martial is not alone in mentioning literary sales - and this is hardly the only reference to the activity in Martial’s poetry - in this poem we can get a good sense of the increasingly important role played by commercial sales of books alongside the continued existence of traditional personal circulation of a work by its author.
To move onto costs, we should probably mention materials. For most of antiquity the majority of book production, whether in bookroll form (as was most common for Classical literature up to the end of the 3rd century CE) or as a codex (as was most common for Christian literature of all periods, Classical literature after the 3rd century CE), used papyrus. Although an older assumption still sometimes circulates about the expense of using papyrus, that’s been quite thoroughly debunked at this point, and we have no reason to think that papyrus as a material was prohibitively expensive for book production.
I should also note though that while there were booksellers, there wasn’t anything that looked like modern publishing. A book’s initial circulation was informal and happened by the individual dissemination of a text by the author’s circle. Additionally, booksellers made their own copies, which means that, like getting a book to copy from a friend, patrons were limited to what their bookseller could access. And there was no sort of central publishing house, and there was no sort of large-scale printing that could then be distributed. All of this was driven by individuals creating a sort of broader-reaching version of what already existed to produce books.
All this is to say that lack of a printing press didn’t prevent the development of a robust literary culture, one whose demands were capable of supporting the sale rather than solely the private distribution of books, but it looked quite different from what we see later.