Historically, Chinese migrants to the US were not from a diverse set of places. The earliest migrants – who in some but all cases became immigrants; most were there temporarily as migrant workers only – came from the Pearl River Delta area. I'm borrowing from some earlier answers I've given on this topic just to expedite the writing process.

At the time of early significant migration to the US, near the end of the 19th century, things in China were not so great, to put it mildly. The 1800s saw a huge number of problems in China, particularly in South China where most migrants were coming from. There were numerous famines (1810, 1811, 1846, 1849), a good bit of piracy on the Southern Coast, plus you had the Opium War (1839-1842) and, starting slightly before the Taiping Civil War, you had the Punti-Hakka Clan Wars (from about 1855 to 1867, and, fun fact, being fought at the time on the Malay Peninsula as well). Then before the Taiping Civil war was over, you had the start and end of the Arrow War, a.k.a. Second Opium War. In addition to all of this (because it clearly isn't enough), you had major economic hardships in the south (which were in large part the thing that set off the Punti-Hakka wars, but certainly not the only thing being fought about) as a result of urban centres exceeding the carrying capacity of the land and as a result of that, large scale migrations throughout the region. Then in addition to all of that, at the same time you also had the Nian Rebellion from 1851 to 1868, catastrophic flooding of the Yellow River in 1851 and 1855, which devastated the grain supplies for the country and killed countless people.

Most of the worst-hit areas were in the South, in Cantonese and then Hakka speaking areas. Similar to coyotes today, there was a lucrative business of sending Chinese to work in areas like the US West both willingly and unwillingly, and these were most active in the areas around Guangzhou, hence the reason that's where most people were coming from. There was already considerable trade happening between Hong Kong and places like San Francisco. In fact, early on in the gold rush, getting your clothes laundered often involved shipping them all the way to Hong Kong and waiting for them to come back, being that that was cheaper than shipping them out east, and local operations had not yet been established. These existing shipping routes helped bring people as well.

For many of these migrants, they were able to work under poor conditions in the US for less money than others were making there, and still have enough money to send back home. They would do this for a year or more, but then often return to China. Few had the intention of actually immigrating. Despite this, the US governments, in particular that of California, were far from keen to have Chinese coming into the country. Due to racism against the Chinese and "economic anxiety", Californians pushed the US government to heavily restrict immigration. It happened slowly, but became near-total.

Initially things were more open. The 1868 Burlingame Treaty, an amendment to the Treaty of Tientsin form ten years prior and one of the Unequal Treaties signed between Western powers and the Qing, allowed for free migration (but not naturalisation) between the two countries. However this didn't last long, and in California there was already a well established record of local anti-Chinese policy, including the so-called Anti-Coolie Act of 1862 which put a tax on Chinese doing business in the state, and before that another tax in 1852 and an anti-immigration law in 1855. Then more substantially, in 1875 there was a law called the Page Act, pushed by the largely-white labour unions, which was meant to be a means of protecting the American labour force from wages that other people couldn't compete with. This was done because the non-Asian population in the US at the time felt that these foreigners were coming and taking their jobs, a sentiment that may sound familiar to Californians today. The Page Act did not outright stop Chinese migration. Instead it was primarily worded to prevent people from being brought against their will. That's all well and good, but it still was an early testing-of-the-waters for more thorough legislation against Chinese migrants.

Quoting from the Page Act:

Be it enacted by the Senate and House of Representatives of the United States of America in Congress assembled, That in determining whether the immigration of any subject of China, Japan, or any Oriental country, to the United States, is free and voluntary, as provided by section two thousand one hundred and sixty-two of the Revised code, title "Immigration," it shall be the duty of the consul-general of consul of the United States residing at the port from which it is proposed to convey such subjects, ... to ascertain whether such immigrant has entered into a contract or agreement for a term of service within the United States, for lewd and immoral purposes; and if there be such contract of agreement, the said consul-general or consul shall not deliver the required permit or certificate.

The phrasing "for lewd and immoral purposes" is significant given the common yet conflicting stereotypes of Asians as both hyper-sexualised and of Asian men as effeminate to the point of being entirely sexless.

That year also saw a Supreme Court case which determined that immigration laws were not the domain of the state, but instead could only be enacted by the federal government. This was in response to California's tendency to limit immigration, but of course taking this decision out of the California government's hands didn't actually do anything to reverse the trend, and California was in no way alone in their attitudes toward the Chinese, and neither was America.

Still, ultimately the Page Act didn't do much to stop immigration from the region, and Anti-Chinese sentiment continued to develop. A few years later, things became formally much more restrictive. I'll skip some of the specifics to keep the length of this under control. 1880 was the start of major legislation at the national level, with more amendments to earlier treaties. In 1882 things got worse as a law was signed into effect in the United States that effectively limited any immigration from East Asia. This is known as the Chinese Exclusion act, which was in effect until it was repealed in the 1940s. It was originally a temporary move but it was later made permanent. This was renewed in 1892, and in various forms again and again until after WWII.

So, how does all this relate to the original question?

Basically what happened was at the very beginning you only had migrants coming from the hardest-hit parts of the South who were still able to manage to get overseas, i.e. also lived near or were able to get to the major ports of the Pearl River Delta. Then, as things shifted and China started moving into the Republican Era, which would have potentially brought more immigrants from more areas, the US laws (and under the US's pressure, the laws of Canada as well), got much more restrictive and effectively prevented large-scale immigration from other parts of China as well as the South.

China is quite diverse today, but the vast majority of that diversity is in the far Southwest in mountainous areas, with ethnic minorities many of whom did not speak a Sinitic language in the 19th century, or people like Uyghurs, Kazakhs, Tajiks and Kyrgyz speakers who were nowhere near the infrastructure to get overseas. There was considerably more immigration from the Northwest to places like Washington DC but that only happened much more recently, and due to the political situation with the Central Government in Xinjiang.

Then in cases such as Min/Hokkien/Fujianese speakers, in most cases the diaspora is really much more contained within Southeast Asia, and is also much older than the 19th century. Few migrated to the US at that time.

So, in short, the greater diversity of China simply wasn't really reflected in the diaspora in America at all, with the vast majority coming from the Guangdong (Canton). Thus why Cantonese is so well known to Americans, but other Sinitic languages are not. There were Hakka migrants, but they were a minority within a minority, and so also were of much smaller number, and still were coming from the same region.

tl;dr: It mostly didn't.

References:

• Chang, Iris (2004) The Chinese in America A Narrative History. Penguin Mass Market.

• Gyory, Andrew (1998) Closing the Gate: Race, Politics, and the Chinese Exclusion Act

• Lee, Erika (2003) At America’s Gates Chinese Immigration During the Exclusion Era, 1882-1943. The University of North Carolina Press.

• Mar, Lisa Rose (2010) Brokering Belonging Chinese in Canada’s Exclusion Era, 1885–1945. Oxford University Press.

• Siu, Paul C. (1987) The Chinese Laundryman: A Study in Social Isolation. Edited by John Kuo Wei Tchen. New York University Press.