I was reading on Wikipedia, that during the time of the Byzantine Empire, the term Byzantine was not applied, and citizens thought of themselves as Roman. So why, after the fall, was the name Byzantine applied? I understand that culturally it had changed quite a bit from the proper "Roman" times, yet the Roman Republic, and Roman Empire also changed quite a lot during its time. So why Byzantine? Why is it not called "The late Roman Empire" or something similar?
Essentially because several successive polities in the west, including the papacy, claimed to be the successors of the Roman Empire, and you can't be the successor to the Roman Empire if the Roman Empire is still kicking. They were still there, though, so they had to be called something.
The Byzantines, however, always called themselves the Roman Empire, and this is something the majority of historians now acknowledge to be the most accurate representation of the truth. However, you'll still catch us referring to it as the "Byzantine Empire" by habit and convention.
The phrase Roman means so many things.
Are you talking about the city of Rome itself? The culture of the people who live in the city of Rome? The culture of the empire of the city of Rome? The empire of the citizens of the civilized Mediterranean world? The people under the empire of the Roman emperor? The culture of the pagan Roman Empire? The Christian Roman Empire? The empire that draws a continuous political inheritance from the empire of the city of Rome? And on and on and on.
I know you (the average reader) have a tendency to want to respond with "well, you KNOW what I mean", but the problem is that while you have a specific conception of what the Roman Empire is (namely pagan high imperial Rome), historians, "Byzantines", late Romans, high Romans, and republican Romans, have a different conception of it themselves, and as historians we have to find ways to shorthand identify the connotations and differences of each stages.
As you can tell, each one of these (which is still a partial list) possibilities changes the scale and scope of what a Roman Empire is. Each one can be quite different.
So why Byzantine? Because even though the "Byzantine Empire" draws a continuous political inheritance from the old "Roman Empire", it is no longer a supraregional spanning multinational multiethnic poly-religious superpower empire as it was in high imperial days. Which is the connotation you get when you say "Roman Empire". Instead, it was in essence, the regionalized Greek Orthodox Christian Empire of Constantinople, the Balkans, and Anatolia.
Unfortunately, Constantinopolitan Empire is a bit of a mouthful. So what's the alternate name for Constantinople? Byzantium. Keeping in mind too, that Byzantine historians frequently recorded the denizens of the imperial city as "Byzantines", so it's not just a phrase we modern historians invented out of nowhere.
Thus we have, the Empire of Byzantium, or Byzantine Empire. It works on many levels to separate the empire that existed after the Arab Conquests from the Eastern Roman Empire and the Roman Empire. Because as mentioned previously, the differences were quite stark, which makes Byzantine Empire a useful temporal marker for us.
Ultimately though, you should know that outside of the historical use of Byzantine, the choices of names for the "Roman Empire" ultimately is a political one, expressing a certain "nationalism" over identity and power. Because to legitimize the Byzantines as Romans is to express the similarity of the later empire with the power of the earlier one (which is why the Byzantines referred to themselves as Romans, and also why Internet wargamers who love to play medieval strategy games are so insistent that the Byzantine Empire be called Roman). And conversely, to separate the Byzantines from the Romans is to express dissimilarity of the later empire with the power of the earlier one (which historians do because despite the political continuity, the cultural, infrastructural, and military continuity was far more sketchy).