Anti-black discrimination and segregation, both de jure and de facto, remained deeply interwoven in the fabric of American society for at least a century after the Civil War (and unfortunately haven't been entirely eradicated today).
However, from what I've read, this doesn't seem to have been reflected in immigration laws from about 1865 to 1965, at least to not the extent that might be expected. I understand that people from predominately black countries couldn't immigrate in unlimited numbers, but that this also applied to people from nearly all non-Northern European countries, which is why I write in the title that black immigrants were apparently never "specifically" restricted. This is in contrast to the Chinese Exclusion Act of 1882 and the Immigration Act of 1924, which limited non-Northern European immigration in general but imposed a blanket ban on all immigrants from Asia.
Given all the formal and informal mechanisms of oppression of black people in the US in full force in the 19th century and much of the 20th century, it seems contradictory to me that the politicians at the top of this structure nevertheless created laws that allowed foreign-born black people including Marcus Garvey, Sidney Poitier, and the Caribbean-born parents of Colin Powell, Malcolm X, and Harry Belafonte to freely immigrate to the US during the period concerned (edit: I'm finding out that many prominent African-Americans have recent Caribbean ancestry that I never would have expected).
In other words, why does anti-Asian racism seem to have made a much bigger dent in US immigration policy than anti-black racism, which was much more deeply rooted and fundamental to how American society was set up?
This is a great question!
Let us start with the Naturalization Act of 1870. This Act allowed for African immigrants to naturalize (i.e., become citizens) for the first time. This was a Reconstruction Era law, at a time when Republicans (then the party of abolition and Lincoln) controlled Congress and passed various laws that enriched the rights of Blacks in America. This sentiment decidedly did not extent to Asians at this time period, who were all but universally seen as outsiders and aliens. The most stark example is Justice Harlan's dissent in the infamous case of Plessy v. Ferguson, 163 U.S. 537 (1896). That case allowed segregation in this country for decades, and Harlan wrote a fiery dissent--but included a remarkably racist passage about Chinese people as compared to Black Americans. To quote:
There is a race so different from our own [i.e., whites] that we do not permit those belonging to it to become citizens of the United States. Persons belonging to it are, with few exceptions, absolutely excluded from our country. I allude to the Chinese race. But, by the statute in question, a Chinaman can ride in the same passenger coach with white citizens of the United States, while citizens of the black race in Louisiana, many of whom, perhaps, risked their lives for the preservation of the Union, who are entitled, by law, to participate in the political control of the state and nation, who are not excluded, by law or by reason of their race, from public stations of any kind, and who have all the legal rights that belong to white citizens, are yet declared to be criminals, liable to imprisonment, if they ride in a public coach occupied by citizens of the white race.
What we can glean from this--and other writings like it--is that white America's fear of the Asian "other" was incredibly strong. Asians were regarded as "unassimilable" even by whites who supported Black rights, and those whites heard no dissonance in that. African Americans were by and large Christian, had been in the U.S. for centuries, and spoke English; Asians were just "different." The history of discrimination in California is particularly stark--various laws were passed restricting the rights of Chinese people, but not those of Blacks. This sense of the "otherness" of Asians explain why Republicans, in the era of Reconstruction, allowed Africans to naturalize, but did not allow Asians to.
By the time of the Immigration Act of 1924, anti-immigration sentiment was very strong, especially against Asians, who were pretty much totally barred from entering the U.S. Africans were technically permitted to enter the U.S., but the quota system that this Act used--2 percent of that country's population in the U.S. in 1890--along with difficult literacy tests mandated by the Act ensured that the number of Africans who could enter was minuscule. The legislators who passed this law would have known that the number of African migrants would be tiny, so it was probably not worth it to enact a total ban on African migration (especially in the face of potential African-American opposition, who were still a Republican constituency at this point in history). Furthermore, the Act placed special restrictions on immigration from colonies; since most African countries were colonies in the 1920s, this further limiting African migration to incredibly low levels, all without a total ban. There just wasn't a need to enact a total ban if one's goal was to limit African migration. A total ban on Asian migration would be worth it, however, since there were a large number of Chinese and Japanese immigrants in 1890, and that population was almost universally reviled by white America.
Of course, just because the Act technically allowed Africans into the country did not mean much practically. Africans accounted for only about 1 percent of the total immigrant population before 1965.
Another important factor to keep in mind about the Immigration Act of 1924 is that it did not restrict immigration from countries in the Western Hemisphere. This explains the ability of Afro-Caribbeans to migrate to the United States in this time period. Why the lack of restrictions against Western Hemisphere migrants? This is likely a legacy of Monroe Doctrine policy, in which the United States asserts its supremacy over the New World. Migration to the U.S. from Western Hemisphere countries would have been acceptable on the grounds that the U.S. wanted to deepen ties with countries in its backyard.
This brings us to the Immigration and Nationality Act of 1965, a product of Civil Rights Era legislation that sought to eliminate racial discrimination broadly. The INA massively opened up immigration to the United States and eliminated racially conscious criteria. It was based on a quota system that prioritized family reunification rather than skilled labor. Despite being a landmark piece of civil rights legislation, aspects of the INA were passed with an eye towards racial/demographic considerations. Scholars generally believe the INA's family reunification system had the effect of alleviating concerns that massive demographic change would occur. The vast majority of families to be reunified in America at the time were white, and it was generally felt that the Act would not dramatically change the demographic profile of the U.S. Specifically, proponents believed the Act would not increase Asian migration--most famously, Attorney General Robert Kennedy testified that Asian immigration was expected to be no more than 5,000 in the first year. Scholars tend to believe the increase in Asian immigration that resulted from the INA was unintended. This is disputed by some, I should note. See Gabriel Chin, The Civil Rights Revolution Comes to Immigration Law: A New Look at the Immigration and Nationality Act of 1965, 75 N.C. L. REV. 273 (1996) (arguing that proponents of the INA knew about, and welcomed, the demographic change that the Act would produce). This viewpoint is perhaps bolstered by some Southern Democrats' opposition to the measure on racial grounds. (On the other hand, one could argue the family reunification system was put in place precisely to alleviate Southern Democratic opposition.)
Sources:
Plessy v. Ferguson, 163 U.S. 537 (1896).
Gabriel Chin, The Civil Rights Revolution Comes to Immigration Law: A New Look at the Immigration and Nationality Act of 1965, 75 N.C. L. REV. 273 (1996).
Gabriel J. Chin, A Nation of White Immigrants: State and Federal Racial Preferences for White Noncitizens, 100 B.U. L. REV. 1271 (2020).
Immigration Act of 1924, 43 Stat. 153 (1924).
Immigration And Nationality Act of 1965, 79 Stat. 911 (1965).
Bill Ong Hing, African Migration to the United States, Assigned to the Back of the Bus, in THE IMMIGRATION AND NATIONALITY ACT OF 1965 (Gabriel J. Chin & Rose Cuison Villazor eds., 2015).