The whole history of this period's relations between the US & UK and China is incredibly complex. This is only the most basic answer. I could go on for pages but I think that's probably not necessary.

In the mid-1800s there were a large number of forced labourers going to the United States. This combined with the growing Yellow Peril after the Opium Wars created strong anti-Chinese sentiment. The ideas of Social Darwinism were taking hold and attitudes toward Asians were not good in the West.

Before things became so restricted, the 1868 Burlingame Treaty was signed between China and the US. It was an amendment to the Treaty of Tientsin of ten years prior, one of the Unequal Treaties signed between Western powers and the Qing. The Burlingame Treaty allowed for free migration (but not naturalisation) between the two countries. This didn't last long, and in California there was already a well established record of anti-Chinese policy, including the so-called Anti-Coolie Act of 1862 which put a tax on Chinese doing business in the state, and before that another tax in 1852 and an anti-immigration law in 1855.

In 1875 there was a law called the Page Act, pushed by the largely-white labour unions, which was meant to be a means of protecting the American labour force from wages that other people couldn't compete with. This was done because the non-Asian population in the US at the time felt that these foreigners were coming and taking their jobs, a sentiment that may sound familiar to Californians today. The Page Act did not outright stop Chinese migration. Instead it was primarily worded to prevent people from being brought against their will. Quoting from the Page Act:

Be it enacted by the Senate and House of Representatives of the United States of America in Congress assembled, That in determining whether the immigration of any subject of China, Japan, or any Oriental country, to the United States, is free and voluntary, as provided by section two thousand one hundred and sixty-two of the Revised code, title "Immigration," it shall be the duty of the consul-general of consul of the United States residing at the port from which it is proposed to convey such subjects, ... to ascertain whether such immigrant has entered into a contract or agreement for a term of service within the United States, for lewd and immoral purposes; and if there be such contract of agreement, the said consul-general or consul shall not deliver the required permit or certificate.

It was a lose definition of who it applied to. China and Japan are clear enough, but "any Oriental country" can be taken pretty broadly.

The same year also saw a Supreme Court case which determined that immigration laws were not the domain of the state, but instead could only be enacted by the federal government. This was in response to California's tendency to limit immigration, but of course taking this decision out of the California government's hands didn't actually do anything to reverse the trend, and California was in no way alone in their attitudes toward the Chinese, and neither was America.

Still, ultimately the Page Act didn't do much to stop immigration from the region, and Anti-Chinese sentiment continued to develop. A few years later, things became formally much more restrictive. Leading up to this there were already significant restrictions places on East Asian immigrants such as not allowing them to naturalise, and also limitations on what type of business they could do.

1879 saw the "Fifteen Passenger Bill" which only allowed up to 15 Chinese passengers on any ship coming to the United States. President Hayes was also strongly opposed to immigration (which he called invasion) of non-whites (which he called the "weaker races"). Still, he ended up vetoing it only based on it conflicting with previous treaties and therefore not actually passable, despite a large majority in the House and Senate supporting the bill.

Then In 1880 the previous treaty between the Qing and the US was modified with the Treaty Regulating Immigration from China, limiting all blue collar workers from immigrating, not just prostitutes and forced labourers as has been the case before. Quoting from the treaty:

the Government of China agrees that the Government of the United States may regulate, limit, or suspend such coming or residence, but may not absolutely prohibit it.

That's the wording written by the US, in case it wasn't clear.

Then in 1882 things got worse as a law was signed into effect in the United States that effectively limited any immigration from East Asia. This is known as the Chinese Exclusion act, which was in effect until it was repealed in the 1940s. It was originally a temporary move but it was later made permanent.

1892 saw a renewal of the Chinese Exclusion act as the Geary Act, which added that all Chinese arrested will be assumed illegal immigrants unless they can give solid proof that they arrived before the passage of the previous acts. What constituted as proof was a certificate of residency.

There's way more. There were many court cases and smaller efforts to further reduce the number of East Asians. It was an ongoing issue that had a slow burn throughout the period. However this should be sufficient to answer your question, at least for the China side of things.

One more thing worth bringing up: Why were so often the target when Japanese, while listed, were less obviously a concern? The two main factors were that Japan had already begun actively developing starting in the 1850s when forcibly opened to the West, and also there were just so many more Chinese than Japanese at the time, as today. It's not that Americans were more positive to Japan so much as they were just less aware of Japanese people as a unique threat to their situation.

tl;dr: The main reason why there was so little immigration to the US from East Asia during the time you mention is because it was effectively stopped by the government and largely supported by the native populations.

See also:

• Gyory, Andrew (1998) Closing the Gate: Race, Politics, and the Chinese Exclusion Act

• Lee, Erika (2003) At America's Gates: The Exclusion Era, 1882-1943

• Lovell, Julia (2011) The Opium War: Drugs, Dreams and the Making of China. (This goes beyond the wars with China & the UK and talks about the building Yellow Terror and attitudes toward Chinese in the West)

• Zhu, Ming M (2010) The Page Act of 1875: In the Name of Morality

^edited ^to ^add ^some ^sources ^/ ^further ^reading.