In Classical Greek usage, the term "barbarian" did not carry the strong negative connotations that it does for us. It was used to describe people who were not Greek. While there is no evidence for the common belief that the word originated from Greeks mocking non-Greek speech ("bar-bar-bar"), it remains one of the possible etymologies of a term that simply indicated "them" as opposed to "us". It did not necessarily mean savage or uncivilised. Indeed, the Lydians and Persians were barbarians known for their wealth and taste. The Egyptians were barbarians known for their ancient wisdom.
Now, I'm not so naive as to assume that you can divide the world into "us" and "them" without attaching at least some pejorative meaning to the concept of "them". This is "othering": stressing the ways in which people from the out-group are different, and framing that difference as something sinister or inferior. In that sense all uses of the term "barbarian" are, by definition, xenophobic. Definitions of us and them are value judgments with cultural and political power to exclude, marginalise, and harm. It is impossible to use a term like "barbarian" without revealing a fear of, or contempt for, the other.
But we should bear in mind that this also applies to the concepts that we use to justify dividing the world into civilization and barbarism. This is not some natural, observable binary, and "advancement" is not a measurable quantity. These are subjective categories that the modern western world has invented in order to formalise and rationalise its sense that westerners are better than other people. These categories draw on ancient discourse about civilised Greeks and Romans versus the barbarians of the East and North, and instrumentalise ancient prejudice in an attempt to make modern prejudice seem justified and rational. In doing so, it makes the Greek concept of the "barbarian" into something it never was: the wild man from the dark forest, the weakling slave of an eastern despot, and so on. Sure, the roots of those stereotypes are visible in the ancient sources. But in many ways the divisions you are drawing between peoples of the ancient world reflect modern prejudices, not ancient ones.
It is impossible to answer a question like this without indulging those prejudices. We might be inclined to think Greeks were more civilised than Thracians because they lived in cities, or because they wote history and philosophy - but that is only because we have been taught that these things, which are not innately better or worse than other ways to live, are the mark of cultural superiority. We have chosen these things as marks of superiority because they fit the Greeks but not the Thracians. On what reasonable grounds could we declare one culture more "advanced" than another, once we realise that all possible measures of "advancement" are deliberately designed to favour the culture we assimilate to ourselves?
The very idea that some groups can be more civilised or advanced than others is xenophobia. We would not even have these concepts if not for our need to feel justified in our xenophobia. In reality, Greeks, Macedonians, Illyrians and Thracians lived similarly in some ways, and differently in other ways; what good would it do to try to rank their ways of life?