I was looking at the Greek letter Chi (X) and thought that must have been confusing for Romans who had a number X. But then I read the Romans took Chi and made it into the letter X we know today. But what happened to the number X they were using?
I think you have the wrong idea about how the Latin language developed. Yes, it can be said that the Romans 'took' the letter Chi and adapted it into their language, but it wasn't as if it was a conscious choice to start using 'X'.
The Romans got their letter X from the Etruscans, who themselves got it from Euboean Greeks. They didn't pick and choose their letters, however. For the first several centuries of its existence, Rome was dominated by the Etruscans, who we today know very little about, though it is very clear that they held considerable influence over Rome between around 700 and 300 BC. The Roman historian, Livy, notes in fact that it was common in early Rome (3rd century BC) for Romans to be bilingual in Roman and Etruscan.
Nicholas Ostler said "the effect of Etruscan on Latin was quite comparable to the effects on medieval English of French" (36), to give you an idea of the scale of Etruscan influence on Latin. For instance, 'Roma', the Latin word for Rome, is an Etruscan word. The Roman system of using three names rather than a name and a patronym also probably came from Etruscan.
So long story short, the Romans got the letter 'X' from the Etruscans when Rome was a small kingdom in Italy under the domination of Etruria. Early Romans learned the Etruscan language, and any confusion between the letter and the numeral 'X' would have been as easy to overcome as the confusion between P and Р for an English speaker learning Russian. After around 500 BC 'X' was a letter in their language which just also happened to function as a numeral, and confusing the two would probably have been the same as the notion of an English speaker confusing O and 0.
Sources:
Ad Infinitum: A Biography of Latin by Nicholas Ostler (Walker & Company)
Die Etrusker: Geschichte - Religion - Kunst by Prof. Friedhelm Prayon (CH Beck)
All of the Roman numerals are also letters. They used context to disambiguate, just as we don't confuse a number like MCMXCVI for an English word.
We have O and 0. Confusing? Sometimes, but context is key?
A few ancient languages used the same characters for their letters and their numbers. Greek did it too, with each letter having a numerical value (their Chi was 600). Though Greeks made it obvious by putting a hash mark after to show that it was meant as a number.