For example, could a regular citizen in Alexandria just walk into the library and take out some scrolls to study or were libraries reserved only for certain classes of people?
edit: also, could those people permitted at the libraries remove materials and take it home with them or were the scrolls only available for use on the library?
That's a great question and one I don't know the answer to! I can, however, give somewhat of an almost-answer for ancient Mesopotamia, specifically the Neo-Assyrian empire.
As far as I know, there's no evidence of public libraries in Neo-Assyrian cities, but there were two institutions that sort of resembled libraries - palace and temple archives. The most famous of the palace archives is the so-called "Library of Ashurbanipal", assembled by the eponymous king as a testament to the quality of his learning and the reach of his empire. You can find a fair bit of info about the contents of this library, but I haven't seen much about how it functioned. Texts were catalogued and organized according to their colophon, but I don't think we know if these texts were publicly accessible or not. I'd be inclined to guess that they weren't, due to the archive's position in the palace, and that it would've been reserved mainly for the use of the royal court (the court supported a number of scholars in different advisory and academic capacities, and they might've made use of some of the text collections professionally).
Outside of the palace, there were certain temples with extensive archives, especially the temples of the god of scribes and learning, Nabu. An excellent source on these temples is Neumann's 2018 "Reading the Temple of Nabu as a Coded Sensory Experience," https://www.academia.edu/37830397/Reading_the_Temple_of_Nabu_as_a_Coded_Sensory_Experience?auto=download&campaign=weekly_digest Part of this paper discusses "specialised knowledge" in the context of the temple archives, particularly regarding a class of educated specialists called the ummanus or "masters" and their possession of secret knowledge. Professions which fell under that category included āšipu, ritual experts; tupšar Enūma Anu Enlilla, scribes of the omen series "Enuma Anu Enlil" (sort of like soothsayers); bārû, diviners; asû, physicians; and kalû, lamentation priests.
Quote:
These experts obtained their elite knowledge by studying a corpus of cuneiform texts that was particular to their discipline. Scholarly texts with preserved colophons dedicated to temples of Nabu and rooms within this god's temples at Kalhu and Dur-Sarrukin with characteristics befitting scribal offices, including tablet assemblages and pigeonholes for storage, establish and undeniable connection between the god Nabu and Scholarship (Hunger 1968: nos. 327-8, 338-9; Robson 2014).
...
Through their regulation of access and movement and of the sensory phenomena of the interior, they indicated that the areas within the temple, especially the cult room, catered to activities of a select, privileged group of individuals who were permitted access.
...
Thus, in effect, the people who would have had the opportunity to engage with the evocation of specialised knowledge within the temple were very likely those in possession of some form of it themselves.
So, access to at least certain collections of texts was restricted to educated professionals in particular fields. I wish I could say more about what degree of access to other sorts of texts was available to the average member of the public - were temple hymns or literary tablets publicly available? Did scribal schools for young students have their own libraries? Literacy was not the norm in most cities throughout Mesopotamian history, but it was surprisingly common in some periods, such that we have extensive records of personal letters, receipts and the like, so could literate people access texts to read for fun or personal study? I've no idea.
For more on restricted knowledge, see Alan Lenzi 2013 "Advertising Secrecy, Creating Power in Ancient Mesopotamia: How Scholars Used Secrecy in Scribal Education to Bolster and Perpetuate Their Social Prestige and Power"
Also Robson 2013 "Reading the libraries of Assyria and Babylonia" https://www.academia.edu/2299200/Reading_the_libraries_of_Assyria_and_Babylonia_page_proofs_
I haven't read that last one, just grabbed it from Neumann's bibliography, but it looks like the book that's from might be precisely what would answer your question more thoroughly! https://books.google.com/books/about/Ancient_Libraries.html?id=3QRaAQAAQBAJ
Also just found this paper, Potts 2000 "Before Alexandria: Libraries in the Ancient Near East" https://www.academia.edu/1959976/Potts_2000_Before_Alexandria_Libraries_in_the_Ancient_Near_East?swp=rr-rw-wc-2299200
As well as Lionel Casson, "Libraries in the Ancient World", and Ryholt & Barjamovic, "Libraries Before Alexandria: Ancient Near Eastern Traditions"
Hopefully at least one of those has your answer!