Were there museums in the age of, say, Alexander? What did they display? When did they become commonplace? Did they do historical research?
The ideology of a museum, specifically art museum did not happen until the French Revolution. Before that, collections were hidden away for those entitled enough to be in an academic institution.
Notion of the Museum was brainstormed as a means to stop iconoclasm and preserve nationalism. To answer your question about what was displayed - objects were specifically that of national importance.
Ennigaldi-Nanna's museum in Ur (Neo-Babylonian Empire, present Iraq) has been called the first actual museum (dating ca 530 BC).