There is an account by Josephus in Antiquities of the Jews, Book XIV, Chapter 4. He writes that Pompey entered the temple and saw all of the ritualistic pieces: including the Menorah, table of shewbread, sacred vessels, and spices/incense. The Roman General respected and didn't remove any of them. As it says, "Yet did Pompey touch nothing of all this; on account of his regard to religion; and in this point also he acted in a manner that was worthy of his virtue."

However, those objects are only part of the temple's outer sanctuary. The record hints that Pompey did enter the holy of holies as well, but doesn't mention what he saw inside. It simply states that he saw "all that which it was unlawful for any other men to see, but only for the High Priests." This somewhat suggests the holy of holies but does not provide details.

Either way, Pompey entered the temple and thus defiled it, but it appears that he respected the objects inside. He then ordered the temple to be cleansed and gave the high priesthood to Hyrcanus.

There could be other records that go more in-depth about the holy of holies experience of Pompey. I would be curious to see them too.