Yes, it is true. The so called "Byzantines" identified strongly as Romans for the duration of their history. It's actually pretty much impossible to overstate just how strong that identity was, at least until the 13th century (I'll get back to that).

They were the Romans, their state was the Republic of the Romans (1) (not the Eastern Roman Empire it should be noted, since the Western and Eastern Empires were formally reunited in the 5th century), their sovereign was the "August Emperor of the Romans"(2), their customs and learning were "Roman culture" and "Roman wisdom"(3), and their capital was "The New Rome"(4). Illustrious men traced their lineage back to Scipio and Aeneas, and the contemporary Romans and their empire were even referred to as "Ausones" and "Ausonia", Greek for "Latins" and "Latium" in a non-lingual sense (5). Comparisons between the Ancient and "Modern" Romans were commonplace in Byzantine histories (6), and the two were seen as the exact same people in general, just in different times.

This confidence of identity was maintained in full force right up until the crusades, after which mounting anti-western animosity caused the Byzantines to define themselves more and more in opposition to the "Latins". Latin was dropped from imperial coinage in favor of Greek, Augustus was barely used as a primary title anymore, and the Komnenian infatuation with the ancient Greek world, unrelated to the crusades it should be noted, lead to a rise in the use of ancient Greek, and a decline in the use of ancient Roman exempla.

In the end, the culmination of that animosity between east and west, the fourth crusade, solidified and magnified the anti-Latin sentiments of the population. It's after this period that we start to see Byzantine authors outright refer to themselves and others as "Greeks" in an ethnic sense, drawing a direct connection between themselves and the Greeks of the ancient world. This is the identity that would later become that of the Modern Greeks. There was still a strong Roman national identity in this period, but it was coupled with the underlying assumption, at least among scholars, that these Romans were Greeks on a separate, more basic, level (7).

On the term "Byzantines", it was first used as a catch all phrase for the Romans of the east by the 16th century German historian Hieronymus Wolf. It should be noted though that "Byzantion" and "Byzantines" were used throughout the Byzantine period as classicising names for Constantinople and Constantinopolitans in specific, probably because the latter two are ridiculously long (8). The designation was never extended to anyone outside of the walls though.

(1) Self identification as "Romans" and the use of "Politeia ton Romaion"(the Republic of the Romans) when referring to the Byzantine empire are present across many texts, such as the Taktika of Leo VI and the Chronographia of Michael Psellos.

(2)"Augustus" was used quite frequently in the pre-Komnenian period. Examples can be found in the aforementioned Taktika, on many coins and seals, and in the History of Leo the Deacon, from which the quote "August Emperor of the Romans" is drawn.

(3) Both from the History of Michael Attaleiates

(4) Constantinople is referred to as "The New Rome" in the Epitaph of Basil II and on a 9th century Patriarchal seal of Photios for a couple examples.

(5) Both from Attaleiates as well.

(6) examples of direct comparison can be found in the Taktika, the history of Leo the Deacon, Kekaumenos's "Oration of Admonition to an Emperor", the Chronographia, the History of Attaleiates, The Alexiad, Theodore Metochites' Lament on the decline of Rome, and many others.

(7) My main source for identity is "Hellenism in Byzantium" By Anthony Kaldellis. A specific example of this duel identity might be found in the aforementioned 13th century Lament of Theodore, where he goes into great detail about the dismal state of the Romans and their Roman empire, on the surface assuming that not much has changed at a basic level, but he then makes an offhand reference to how an ancient author was venerated "By the Greeks up to the present day".

(8) Examples can be found all over the place, like the history of Leo the Deacon, the Chronographia, History of Attaleiates, Etc.

I've found an answer which seems to answer the first part of your answer about referring to the Byzantines as such :

https://history.stackexchange.com/questions/35526/why-did-the-term-byzantine-empire-enter-common-usage-instead-of-eastern-roman/35534

Essentially, Hieronymus Wolf (1516-80) was the first known writer to refer to the Eastern Roman Empire as the Byzantines.

As for Catholic Europe and the Byzantines themselves, the westerners (or 'Latins' to the Byzantines) referred to the Byzantines as the 'Greeks', primarily because they saw the Holy Roman Empire as the true successor to the Roman Empire, or simply because of religious and cultural differences.

Historians referred to the Byzantines as such because their capital was in Byzantion (and later, of course, Constantinople) and not in Rome - it is reasonable to suggest that Catholic Europe also may have thought this.

Geoffrey of Villehardouin, a member of the Fourth Crusade, and Ramon Muntaner, the primary writer on the Catalan expedition, both refer extensively to the Byzantines as the Greeks, whereas Byzantine writers such as Pachymeres refer to the Byzantines as the 'Romans'.

The answer I linked covers this in a fair bit of detail, so I have tried not to simply replicate this - so do read it.

Every time this question comes up I have to post this very great answer by /u/Ambarenya so here goes:

This is a good question, and one which pops up quite often, usually in the context of whether the term "Byzantine Empire" is legitimate terminology. Quite simply, it seems that the term "Constantinopolitan Empire" is often not used due to its length and complexity of pronunciation (although you will see a few die-hard believers in it). For most people, I think the terms "Eastern Roman Empire" and "Byzantine Empire" are vastly preferable to it. Although many argue that the term "Eastern Roman Empire" is the best terminology, I do not personally follow this viewpoint, preferring "Byzantine Empire" instead. I find the argument that "Byzantine Empire" being somehow offensive or illegitimate is rather disingenuous, because arguments could be made on the flipside as well. I wrote a rather lengthy discussion of this in a post some time back which I will reproduce with some slight modifications, to elaborate on why the term "Byzantine Empire" is not illegitimate, and not really any worse than "Eastern Roman Empire", for a number of reasons. Ultimately, it is up for you to decide, because scholars themselves seem rather hung up on it. I am simply presenting an argument based on the evidence. Take it as you will.
(In response to a concern about the inaccuracy of the term "Byzantine Empire")
To start this rebuttal, we will begin by stating that it is incorrect to say "they called their state 'The Byzantine Empire'". Yes, there was never a "Basileia Byzantion". Yet, what muddles that statement is that the term "Byzantium", metaphorically, was quite regularly used to refer to both the city of Constantinople and, in what seems to be a synecdochal way, the Imperial state. (The latter, essentially implying a use no different to the way that "Venice" referred to the "Serene Republic of Venice" or that "Rome" referred to the "Empire of the Romans". It was just another commonly used name, a shorter way to represent the same thing). For a few brief examples from primary sources on this topic, see [below].
And we observe this because the Byzantines were aware of the paradox that they no longer possessed Rome, yet took their name after it. It pops up as a cursory topic in several Byzantine texts, perhaps most directly in the Suda itself (a Byzantine Encyclopedia/Lexicon dating from the 10th-11th Century), which I will recount at the end of this post. Despite falling outside of Imperial control around the 8th Century, Rome was considered to be the dominion of the Eastern Empire, contemporaries believed it was simply in a temporary lapse from their control. They believed that one day, the city would be returned (and it almost was, once in 1025 under Basil II, and again in the 1150s under Manuel I Komnenos). However, we must also remember that Rome was not the most important city to the Medieval Byzantines, Constantinople (Byzantium) was, and to them, it was the center of the world (and many outside the Empire believed it too). After all, was it not declared the "New Rome" by the first Christian Emperor of the Roman Empire? Many Byzantines understood that their lineage descended from Augustus through Constantine, but that Constantine was the one who gave them their great city and their identity as Christians. Constantinople was also far more grand in recent memory in that it sat at the "crossroads of the world" and was virtually impregnable, whereas one of the principal reasons that Rome declined was that it was comparatively in a poor spot to support the trade economy and navy of a sea-faring Empire. Furthermore, at least early on, the concept of dominus was better understood in the East than in the West, due to the prevalence of past Empires that supported similar systems (i.e. Persia and the Macedonian successor states). In these senses, the very definition of Rhomaios had been changed by Late Antiquity. It no longer meant "Roman" literally, but more definitely "a Greek-speaking citizen of the Christian Roman Empire centered at Constantinople".
This definition had likely been accepted since at least the 7th Century, when Heraclius changed the official language of the Empire to Greek and heavily reinforced the concept of the Christian Empire (which was only further reinforced in the following centuries due to the Arab Invasions). Indeed, this is why it is said that Byzantines were offended by being called Greeks ("Hellenes") in later medieval letters, because, to them, it implied not only that they were not the rightful heirs of the Roman Empire (titles to which those "dastardy Franks and [later] Germans also laid claim"), but that they were also not true Christians. But the term "Byzantine" avoided that, because Byzantium was widely known as the old name of Constantinople, it allowed Byzantine authors to remind themselves of the merging of both their ancient Greek (from the old Greek colony) and Roman heritage (because Constantine refounded the City as the new capital) into one cultural image that promoted the scholarship and learning of ancient Greece with the military prowess of the Romans, while also tying in their descent from Constantine. Based on the wide use of the term in many great texts from the middle Byzantine period (including the Alexiad, the Chronographia of Michael Psellos, the Annales of Niketas Choniates, among many others), the word was not at all seen as derogatory, else they wouldn't have used it.
We see, however, that after Constantinople was lost in 1204, and again especially after 1453, the Rhomaic side of the claim could not be as well justified, and so it seems to me that part of the reason you see the rise of use of the term "Hellenes" in later documents (starting perhaps during Choniates' time in the 12th-13th Centuries) is directly due to the weakened state and eventual loss of their once-great City.
But returning to the point I made earlier, here is one of the entries in question, from the Suda Encyclopedia, which dates from the late 10th-early 11th Centuries AD. It highlights, in particular, the Byzantine position on the matter of Rome and Constantinople (under entry "Konstantinoupolis"), which gives us a rare glimpse into their perspective: Constantinople stands as far above all other cities as Rome appears to surpass her; and to be awarded second place to Rome seems to me to be far better than to be named the first of all the others. Three hundred sixty years had passed for the elder Rome since the reign of Augustus Caesar, and the end of her days were already in sight, when Constantine the son of Constantius took hold of the sceptre and founded the new Rome. From the foundation of the new Rome, to the time that the Porphyrogenneti Basil and Constantine† held the scepter of the Romans, the number of years is ______‡. ‡ The text breaks off here, presumably because it was to be filled in upon the death of either one, or both, of the Emperors. The exact date is not terribly important, because we know from the mention of both Basil and Constantine (r. AD 976-1028, inclusive of both) that it must be "about 650 years".
† Although the original editor of the text supposed this meant Basil I (AD 867-886) and his grandson, Constantine VII (AD 913-959), I personally pointed out that this almost certainly instead refers to Basil II (AD 976-1025) and Constantine VIII (AD 1025-1028), due to the fact that it says both of these rulers were "porphyrogenneti" (Basil I was not a porphyrogennetos), and because, fittingly, Basil II and his brother Constantine VIII reigned jointly, and were depicted holding the sceptre together on their nomismata [below]. This issue has been corrected in the latest iteration.

Examples of chronicles of the empire referring to themselves as Byzantine here
The nomismata here

For /u/novashadow1324 and /u/Diocletian335 as well.