As you point out, originally the term βάρβαροι (barbaroi) meant "all that are not Greeks," supposedly bar-bar is the Greek onomatopoeia for incomprehensible speech. It would be like if we called non-English speaking foreigners blahblahians. It was a term especially applied to the Persians and Medes and after the Persian Wars it took on the additional meaning of "outlandish" (which you might see in the works of Aristophanes or Thucydides). By the Augustan age, it has taken on this ideological slant and is used to refer to peoples that lack the accomplishments of Greco-Roman civilization.