I cant speak for what happened generally in the USA, but I do know that when Vivekananda visited the USA in 1893-7 and later in 1899-1902 he was repeatedly called "Negro" and "Nigger" in the streets, which caused him much bafflement and confusion. Luckily he had to stay with rich, liberal women (he of course could not stay in hotels because of his color) and was able to speak with them on more or less equal terms. He got a much more welcome reception in the UK where people knew more about India and Indians.