Africans were able to voluntarily immigrate to the United States after the Civil War ended, and were eligible for naturalization (i.e., to become citizens) after the Civil Rights Act of 1870. However, a variety of factors made actual African migration to the U.S. pretty rare until the passage of the Immigration and Nationality Act of 1965. Even if Africans wanted to come to the U.S., it would have been very hard thanks to these barriers.

One factor, of course, was the expense and difficulty of actually getting to the U.S. Migration would only have been available to a small class of elite, well-educated Africans. The law also made migration by Africans extremely difficult, even if it was technically allowed. The Immigration Act of 1924, passed during an era of high xenophobia in America, imposed incredibly strict quotas on migration from Eastern Hemisphere countries (and a total ban on migration from China and Japan). Each country got 2 percent of that country's population in the U.S. in 1890. Additional restrictions limited immigrants from colonies. There were also difficult literacy tests required of immigrants. The effect of all these restrictions was to make African migration to the U.S. incredibly rare and difficult--since African countries had very few immigrants in the U.S. in 1890, were mostly colonies, and because of literacy tests, very few Africans could actually permanently immigrate to the U.S. Here is some data:

In real numbers, this means that in the 1920s, for example, of 4.1 million immigrants, 6,286 were from Africa; in the 1950s, of 2.5 million immigrants, 14,092 were African.

Pretty low numbers!

As for the distinction between sub-Saharan and North Africans: sub-Saharan Africans would have been eligible for naturalization as a result of the 1870 Act. For North Africans, it depended on whether they could show themselves to be "white." Whites had been eligible for naturalization since 1790; black Africans since 1870; and those descended from the "indigenous" people of the Western Hemisphere since 1940. North Africans would have to convince a court that they were "white" under the law, which usually involved a good bit of scientific racism and hand-waving about the common understanding of what it means to be white (usually based on appearance). A Yemeni Arabian of dark skin was not considered by white, at least by a district court (In re Hassan, 48 F. Supp. 843 (E. D. Mich. 1942). I will try to find a case dealing specifically with a North African, but frankly, these cases are wildly inconsistent, and any given ruling is unlikely to mean a whole lot to another immigrant of slightly different skin tone or from a slightly different background.

One exception to the general hostility of the U.S. system to African migrants was African students. While technically students were not (and are not) considered immigrants by the U.S. legal system, they were nevertheless allowed to come and study at American universities. The U.S. saw this as a key foreign policy investment--have these African students study in America, and they will leave with positive views of the country when they go back to lead their countries. In 1949, there were 888 African students in the U.S. In the late 1950s and early 1960s, civil rights leaders in the U.S. organized "airlifts" of African students from across the continent to study in the U.S. Relatively few of these students, though, would go on to become citizens.

The passage of the Immigration and Nationality Act of 1965 ended the race-conscious immigration system that had existed since the 1924 Act. It set up two big pools of immigrants--those coming for family reunification, and those coming for employment. At least initially, few Africans could take advantage of this system. Very few Africans had family in the U.S., and the pool of Africans with the education and skills demanded by U.S. companies was pretty shallow in 1965.

In the decade from 1960 to 1969 (before the full effect of 1965 Act could be felt), 3.2 million immigrants became lawful permanent residents of the United States. Of that figure, more than a third (1.13 million) was from Europe. Some 358,000 were from Asia, and more than 440,000 were from Mexico alone. Less than 1 percent of the total (23,780) was from Africa, most of whom were from Egypt (5,581), South Africa (4,360), and Morocco (2,880).

As we can see, very few Africans came in the first decade of the INA, and most were not from sub-Saharan African countries (with the exception of immigrants from South Africa, but many of those were likely white). However, these numbers slowly began to increase as more and more Africans got the education/skills desired by U.S. companies, got through the system, and brought along their families with them. Some numbers:

In the 1970s, Africans numbered 80,779 of the total 4.5 million, and in a typical year such as 1987, there were 15,730 Africans of the total 601,516 immigrants. Today, more than a million immigrants enter the United States annually, and approximately 100,000 are from Africa.

African immigrants to the U.S. in the time period after the 1965 Act came for largely the same reasons that any other immigrant group does what they do--seeking opportunities, escaping oppression/instability at home, and earning money to send back to relatives in the mother country.

Importantly, the prospective African immigrant would not have seen the U.S. as the only option. Ties to former colonial overlords in Europe would have been strong. If you were from the former West French Africa, France would have been an attractive option; if from a former British colony, Britain would have been a place you considered. Linguistic, cultural, and economic ties to the former European overlord may have made it easier to move there than to the U.S., especially as immigrant communities from colonies sprung up in those countries, making the immigrant experience a bit smoother.

Sources:

Immigration Act of 1924, 43 Stat. 153 (1924).

Immigration And Nationality Act of 1965, 79 Stat. 911 (1965).

Bill Ong Hing, African Migration to the United States, Assigned to the Back of the Bus, in THE IMMIGRATION AND NATIONALITY ACT OF 1965 (Gabriel J. Chin & Rose Cuison Villazor eds., 2015).

Charles Gordon, Racial Barrier to American Citizenship, 93 U. PA. L. REV. 237 (1945).