The short answer is that chariot games were expensive to hold and maintain, the long answer is that chariot racing's decline was a slow and gradual process which took place as the Roman Empire's revenues decreased.

Chariot games first start declining in the Western Empire during the 5th Century, as central authority weakens and wealthy landowners become less wealthy Hippodromes and Circuses become more and more impractical and expensive to maintain across many Western provinces.

There were some exceptions though, even in the West, as said this was a process. Italy, or atleast the Circus Maximus in Rome would still be used during Odoacer and Ostrogothic rule. Consuls would still sponsor games in their honor and Theoderic the Great would even sponsor a chariot race there during his visit to Rome.

The Gothic Wars however would devastate Italy both financially and demographically, as a result chariot racing became impractical even in Rome. The last chariot race held in the Circus Maximus was by Totila in 549 following his siege of the city. When Circuses fell out of use due to the collapse of a secular authority in Rome and the decline of wealthy landowners there was not much else they could be used for. As a result Circuses would quickly fall into decline, and the stones would be quarried and used as spoila.

In the Eastern Empire however the story is quite different. The Eastern Empire was relatively unaffected by the crises of the 5th Century and as a result there is little to suggest chariot racing there saw a large decline. The Eastern Empire still had plenty of wealthy landowners, Senators, Consuls and a central government which could maintain the Circues and sponsor games.

This changed around the 7th Century. The 7th Century was a traumatic century for the Roman Empire as the Arab Conquests saw them lose Egypt, Syria and North Africa to the Muslim Caliphate. This destroyed the Empire's revenues and left it as a rump state. Many cities fell into deep decline due to constant raiding/warfare and either shrunk dramatically or dissapeared entirely. The only city which managed to avoid such large scale decline was Constantinople, which maintained itself thanks to it's position as capital, it's defensible nature and strategic location in terms of trade.

As a result chariot racing in much of the Eastern Empire dissapeared, but in Constantinople it still thrived. However due to the Empire's reduced circumstances in the 7th Century wealthy landowners became less and less influential. The chariot games would therefore gradually become almost exclusively financed by the Imperial Family. Chariot teams like the Blues and Greens would remain, but due to the Imperial office's increased grip on the Hippodrome some scholars have compared the chariot racing of later eras as an extended part of the Imperial ceremonies of the Great Palace. The chariot games would commonly be used as propaganda for the reigning emperor during this time, some games may have even been rigged in their favour.

But alas, even in Constantinople the ancient games would fade, despite hanging on for so long. Constantinople would remain a wealthy and unconquered city for centuries, until 1204 when it was sacked by diverted Crusaders in the 4th Crusade. The Crusaders carted away almost anything of value and stripped as much of the city as possible. The city also was set on fire more than once accidentally.

The Crusader having sacked the city went about partitioning the Roman Empire, setting up a Latin Empire of Constantinople. The Latin Empire ended up being a dismal failure, the centralized Roman bureaucracy was replaced with less efficent, more feudal governance and due to military losses the Latin Empire's revenues collapsed.

It's during this time that the Hippodrome finally falls into ruin, the last survivor of the Ancient Circuses. When Constantinople was recaptured by the Roman Empire in 1261 they made no effort to repair it, most likely due to decreased revenues.

If you have any further questions or need more details, feel free to ask.