Put simply, it's because the growing anti-Asian sentiment of the 19th century was in many ways a result of competition among labourers, and the legislation was primarily pushed through by labour unions.
A short timeline:
In 1868 the Burlingame Treaty encouraged Chinese immigration to the US, and things like the Boxer Indemnity funds which also resulted in Chinese students coming to the US. Then in 1880 the Angell Treaty basically turns this around and starts to restrict immigration, with the justification that it's just a modification of the earlier treaty, and China can just deal. Formally, this is the beginning of anti-Chinese legislation on a national level. What happened between those years was that a larger number of Chinese were heading to America and working as manual labourers, and since the 1800s saw considerable hardships in China (the Taiping Civil War, the Opium Wars, the Hakka-Punti Wars, famine etc), working a crap job with low pay in America still provided considerably greater income to support your family in China than working in China would. Adding to this, most of the Chinese coming at that time were not intent on staying for long. They were migrant workers, not immigrants (with exceptions of course). They'd go to the west coast, work some years, head back to China. Sometimes returning later with older sons. Rinse & repeat.
This meant that they were also living incredibly cheaply, in temporary housing, or those living in what was becoming Chinatown in San Francisco, in squalor.
So you have a large group of people who dress strange (men with long hair and wearing "dresses" was often pointed out) from a "weaker race" (race-based interpretations of social Darwinism were big on both sides of the Pacific around that time), who were unassimilable, who ate weird foods, and then they come here, and take the jobs working longer days for less money, people started to get angry and their already-present xenophobia was amplified.
At this time, labourers were the focus, because that's where there was the conflict. Merchants, not so much. Everyone needs their clothes washed, after all, and it was merchants that were sending your shirts off to Hong Kong instead of Boston (or wherever) to get cleaned and brought back a couple months later (before people started setting up laundries in California of course). The merchants weren't in competition with the white labourers.
The same thing happened with the Immigration Act of 1924 which explicity banned Japanese because, again due to economic competition, there was plenty of hostility growing in the two decades before then over things like Japanese-run restaurants selling meals for half the cost of what the restaurants run by whites would sell at. Prior to this, at the turn of the century, Japanese were seen in a much more positive (but blatantly fetishised) light.
So that's the short answer.
Let me know if you want me to expand on anything, or if you'd like book recommendations. I'm at work right now so it might take me a few hours to respond.
Also I've answered some other related questions which might interest you, and a lot of the sources in those posts apply here too: