Perhaps I'm wrong, but I can't see Asian people's skin as yellowish, not anymore than a causasian at least. So I wonder how the racist stereotype came to be, and if there's any specific significance to the color.
Edit: sorry about that typo in the title, I can't change it now.
Not to discourage other answers or discussion, but you might want to check out the answers in these posts: