There are three points I would raise here.

First, is that while characters are very difficult, because they're so complicated and there are so many of them (to read 90% of what you'll see in a newspaper, you need to learn 2000 characters), they do have certain advantages. Because they are based primarily on word meaning, not pronunciation, you can use the same characters without the same language. Now, you might point out that English and French use the same alphabet. But, if you've never learned any French, and you see "le chet" you're going to have absolutely no idea what it means. Similarly, a Frenchman who's never learned English wouldn't be able to read "a cat". On the other hand, if you're a Japanese or (ancient) Korean person reading a book written in Mandarin, you can read 猫. A Chinese person pronounces this character as "Māo", while a Japanese reader pronounces it "Neko", but they can both read it. So, for societies with many regional dialects, a character-based system is actually very useful, because it allows people from different regions to still be able to read the same things. Consider that China has many different dialect/languages, which can at times be utterly incomprehensible. But both Mandarin and Cantonese use the same characters. A book in Cantonese isn't easy for Mandarin-speakers to read, but they can understand a lot of it. So characters were very useful for the Chinese bureaucracy, and also useful for peripheral states that wished to engage with Chinese culture. Of course, it wasn't so important for states like Korea or Japan to be able to read Chinese books, which is why they developed their own characters.

Secondly, writing was not a mass skill. It was a specialised skill for bureaucrats and officials, who had to labour for years and pass difficult exams to attain office. Thus, being able to write characters, especially the more complicated ones, was a prestige skill. The officials in charge had no great interest in changing their entire writing system to benefit those who were finding difficult what they had learned. (It's of important note that after the Communist Party took power, with its attempts to educate the entire peasantry, characters were significantly simplified.)

The final point is a matter of tradition, and links to the past. With its Confucian ideology, pre-modern Chinese culture placed great emphasis on its traditional writing system, and even today, with Chinese students learning pinyin (and thus English characters) first and then characters, there have been no significant attempts to change Chinese into a phonetic alphabet. Characters are a signifier of Chinese culture and tradition. In the modern period, other than the Cultural Revolution, Chinese schoolchildren have still been encouraged to practice calligraphy and practice characters as a practice that is both seen as deeply traditional and prestigious. The difficulty of characters only makes them more appealing, as a sign of a good education. To change the character system would be to seperate China from its literary past, a past that stretches back more than 2 millenia.

Edit: Corrected my poor French.