This isn't something I had heard about previously, but your stories prompted me to try and chase down this rabbit hole. I'd love to hear some more in-depth folk tales you've heard, because the rise of these kinds of stories is really fascinating.

Anyway...what really happened?

Sadly, the online records are spotty, and that's what I'm relying on right now. But here is a little to be learned.

The rise in Chinese population in SLO didn't happen until after the 1870's, a while after the California Gold Rush had died down. There aren't a lot of good records to indicate what all the local Chinese were working on, but there are some records that many would have been working on building the nearby railroad. Between railroad and other construction jobs, such as building piers, the Chinese would have gone back to SLO as a home base. They also likely would have been working in fisheries, farms, stores, and laundries, as Chinese were elsewhere along the California coast. For more specifics, you can read this page about Chinese contributions to San Luis Obispo: http://www.wolfcreekarcheology.com/files/ChineseSLOcontributions.pdf

This is where the online resources really break down. In the SLO Public Library, there exists a copy of “A History of Chinese Labor in San Luis Obispo County and a Comparison of Chinese Relations in the County with the Anti-Chinese Movement in California, 1869-1894.” This is what you want--from the articles I've found that reference it, it tells the story of why the Chinese came, and similarly why they left. All I can gather is that in the wake of anti-Chinese sentiment throughout California, the violence that arose in SLO was uniquely bad, and eventually the town just went ahead and rounded up all the Chinese and sent them out of the county, threatening to hang them if they ever returned. I'd have to be able to read that piece I mentioned earlier to get the gritty details, but sadly it has never been digitized.

I was able to find some information about the archaeological dig that was contracted immediately previous to the construction of that parking garage you mentioned, which was built on top of the old Chinatown. They have a whole website here, and there was a news article about that dig (as well as a brief history of the site) here.

Sorry I wasn't much help, but hopefully you have a little better idea than before. And hopefully the San Luis Obispo Tribune will someday add its historic volumes to the California Digital Newspaper Collection (!) so we can dig through some old articles.

I also grew up in a small Northern California town where the Chinese were also driven out in the late 1800's.

Ours were not the only towns either. This was widespread in the west and is not something we are taught in history class.

If you are interested in learning, here is a good book on the subject.

http://www.amazon.com/Driven-Out-Forgotten-against-Americans/dp/0520256948

Pfaelzer, professor of American studies, reveals one of the most disgraceful chapters in American history--the purging of thousands of Chinese immigrants in the Pacific Northwest and Rocky Mountain region between 1850 and 1906. Drawing on newspaper accounts, diaries, legal pleadings, and photographs, Pfaelzer retells the story of the horrific purge of the Chinese. Testifying in their own words, Chinese businessmen recall being driven out of their shops, while women tell of being forced into prostitution; they were driven from gold mines, orchards, and small towns in the booming West. The Chinese responded with defenses from boycotts to lawsuits asking for reparations, challenges to police harassment, shipments of arms from China, and pressure on the Chinese government to intervene. Pfaelzer also catalogs the racist images of docile and dirty Chinese subject to lynchings, night raids, murder, expulsion, and deportation. She compares the expulsions to those in Nazi Germany, as well as modern Rwanda and Bosnia, and puts the Driven Out campaign into the broader context of American racism. Vanessa Bush