The short answer is: We don't really know.

The long answer:

Round shields had always been in use, really. Wide ovoid round shields are evidenced in the 1st-2nd centuries AD for legionnaires, they're just not nearly as common in the artwork. They remain in use up until about a century after the invention of the Kite shield (which, based on what depictions I've seen, seems to be a Roman or a Khazar invention) in the late 800's AD. After that only smaller, Rotella-sized ones are in use.

The precise reasons for these changes are largely unknown, but most people think it has a lot to do with the fact swords became longer. Spathas are harder to maneuver with a rectangular shield in combat than the Gladius, and so we think that the development of the Spatha led to the switch to large, ovoid or circular shields. The Spatha requires more room to maneuver and corners get in the way. The same can likely be said for the switch from the ovoid shield to the first almond-shaped kite shields. The art might correspond with this since we start seeing evidence for a fencing system reflected in middle Byzantine art starting right around 880 AD onwards.

Rectangular shields do not completely fall out of use. The word Thoureos is used up until the 10th century, and we have two depictions of them as late as the end of the 4th and the end of the 5th century AD. It seems in areas where the strip method of construction was more viable than planked construction, or where wicker construction was common, they still stuck around but were not the dominant shield form.