My completely unsubstantiated guess, as an American, is that it's just an uncomfortable political issue, because some demographic is going to see it as an anti-American red flag. There's nothing for a politician/civil servant to gain.
It's a complicated question, but one major factor was that France decided not to invite the US to the Conference on Definitive Weights and Standards in 1798. It did so out of a growing fear of the young US` growing ties to England. The formal, current system was many iterations later, but this snub arguably ensured that the US was not involved in the discussion until much later.
As a result, the US remained on its older system for its formative years (until 1866). The simple momentum of the system by that time, combined with the US` relative isolation compared to European entities, ensured there was simply never much pressure to change. For example, European nations needed to be able to trade, send mail, lay train lines, etc. across many nations at once -- the US, conversely, borders only two nations, and had comparatively minimal interaction with them.
In modern times, I would argue that the country's relative predominance in geopolitics ensures that the US has been able to largely resist pressure to change, outside the academic community.