From what I've read, the Kaaba was dedicated to Hubal, but the Muslims eventually took over. That means at some point, they must have had a different prayer than what we know of today, or am I wrong?
There was a period in time when Muslims prayed towards Jerusalem.
On 11 February 624 CE, during the noon prayers, Muhammad received a revelation to change direction towards Mecca.
The Quraiysh dedicated the Kaaba to a pantheon of idols. Hubal was definitely one of the big ones but it was never dedicated solely to him.
As far as the prayer, the Ka'ba is simply the direction Muslims pray in. The presence or absence of idols never had a bearing on that. The early Muslims prayed facing Jerusalem and, living in Makkah, would often position themselves so that they ended up facing both. In Madinah, that was no longer possible and they prayed facing Jerusalem with their backs to the Ka'abah for about a year until the direction was changed.