If not, what did they think about them after the conquest of Egypt?
The Pyramids were well known in the Arabian Peninsula by the time of the Prophet. There was even an Arab saying:
"All the world fears Time, but Time fears the Pyramids."
While this saying can only be traced to the 12th Century or so, certainly Arabs in general knew about these structures much earlier, due to trade networks in the Middle East.