There's Ennigaldi-Nanna's museum from the late Neo-Babylonian Empire, around 530 BCE in the city of Ur.

Ennigaldi-Nanna (aka Bel-Shalti-Nanna) was the daughter of the Nabonidus, the last king of Babylon, and high priestess of the city. The museum she curated was adjacent to her temple and contained artifacts from the prior two millennia of Mesopotamian history, with some of the oldest dating back to the Ur III dynasty, around 2100 BC. They were even labeled, with trilingual clay cylinders describing the artifacts.

Nabonidus himself exhibited a preoccupation with ancient history as well: he excavated the foundations of the Ebabbar temple in Sippar and had it restored, documenting his discovery of foundation deposits from Naram-Sin of Akkad and a statue of Sargon of Akkad (23rd-24th centuries BCE). Nabonidus seems to have used his interest in history to aggrandize his political legitimacy, having restored the works of his "royal ancestors", and to promote religious reforms related to his worship of the moon god Sin.

Previously mentioned on AH here: https://old.reddit.com/r/AskHistorians/comments/1gx5q8/what_is_the_earliest_example_of_what_we_could/ (the [deleted] poster is me, 8 years ago! weird)