It was! The name "Byzantine Empire" was not used by the Byzantines themselves, or by their neighbors. The Byzantines called themselves Romans, or sometimes Hellenes (after the 4th Crusade the Byzantines promoted their Greek identity more to further differentiate themselves from the Catholic church). The word "Byzantine" would only be used to describe the city of Constantinople itself, not the Empire at large.

This is reflected in Turkish/Ottoman holdings in Anatolia as well. The Seljuk Turkish sultanate in Anatolia was called Rûm, which is derived from the Arabic word for Rome. After the fall of Constantinople, the Ottomans employed an administrative system of "millets", under which certain religious and ethnic groups were given different treatment. The millet for greek christians was called millet-i Rûm, meaning the Roman millet, so even under the Ottoman Empire former Byzantine subjects were defined as Romans, not Byzantines.

So why use the term "Byzantine" at all, if historically it was always just called Rome? Well... mostly its just an invention used by historians as a convenient way to differentiate the primarily Greek-speaking, Christian, Constantinople-based period of the Roman empire from the primarily Latin-speaking, Pagan, Italy-based period of the Roman Empire. The Byzantine Empire was indeed an extension of the Roman Empire into the middle ages, but it is useful for historians to have two different terms to describe the different periods of the Empire since the culture and scope of the Byzantine Empire was so different.