I can offer only partial insight as it pertains to Hampton University and Tuskegee, both of which are often counted among the "Black Ivies," and this insight only extends up until about the end of the 19th century. And my short answer is this: yes, for much of the 19th century, there was a fairly large education/experience gap between white universities and HBCUs.

First though, some general notes: though there were a few black colleges (they wouldn't be called HBCUs until the Higher Education Act of 1965) operating prior to the Civil War (Cheyney University (which was originally known as the Institute for Colored Youth) & Lincoln University, both in PA, and Wilberforce in Ohio, most were established after 1864. (Hampton was founded in 1861, Tuskegee in 1881, more on that later.)

In fact, the vast majority of HBCUs were established following the 2nd Morrill Act of 1890. The 1st Morrill Act of 1862 alotted land in each state for the purpose of establishing a college/university (these are the land-grant universities). The problem was that the majority of states in which such universities were established still had strict segregation policies, even following the Civil War and Emancipation. Consequently, the 2nd Morrill Act called for the further establishment of colleges and universities that could serve black students.

There were (as in the instance of schools like the Cheyney/ICY, Lincoln, Wilberforce, Hampton, Tuskegee, etc) private college/university options for black students. Regardless, however, all of these schools (from the teaching/administrating end) faced several problems:

1. There weren't a lot of college-ready black students.
2. Following Emancipation, the social order (specifically as it pertained to industry and commerce) was totally upended.

These two statements are polite ways of saying that freed folks, who once had been slaves, had been denied access to any sort of education while they were enslaved, and now that they were no longer slaves, tons of large and small businesses (i.e. plantations and new textile manufacturing plants) no longer had access to a compliant, uneducated, and free workforce, (i.e. slaves.)

Consequently, a couple of things happened as it pertains to your question:

1. Many black colleges/universities offered remedial programs that helped students become college-ready.
2. Many black colleges/universities had specialized, dedicated programs devoted to training primary teachers. The idea was that these teachers would then go out into their home communities and establish primary schools that could better prepare (make "college-ready") future generations of black students.

Now, these remedial programs and normal college initiatives (i.e. teacher-training initiatives) were fairly common in many (if not most) of the white colleges and universities in the US, because a lot of the white population faced some of the same "problems' as the black population. Specifically, most poor, Southern whites were similarly unprepared for college because schooling was often nearly as scarce for poor whites in the South as it was for black slaves.

The remedial and teacher-training programs in Hampton and then Tuskegee were especially prevalent, and these two programs had a bizarre twist to them. Hampton was founded deep in Confederate Virginia in 1861 (in the middle of the Civil War) by a fellow named Samuel Chapman Armstrong. Armstrong was not, as you might expect, a champion of justice or equality. In fact, just the opposite. Chapman recognized the "writing on the wall" in regards to the end of slavery, but he determined that there might still be a way to maintain the social order of a slave system without having actual slaves. He reasoned that if you could teach blacks to accept their station in the US social order, then you could still employ them for very cheap. In this way, the South could satisfy the complaints of the Union by eliminating slavery in name only, while still maintaining a rigid social structure that kept blacks as the laboring bottom and whites as the profiting top. (This is all very simplistic; broad strokes).

So, Armstrong established Hampton with the intention of teaching blacks how to be docile, subservient workers. In addition to things like reading, writing, and arithmetic, Hampton included plenty of "courses" related to "practical arts" such as woodworking, bricklaying, farming, domestic work, and so on. Moreover, if a student could not afford to pay Hampton's tuition, Armstrong allowed students to attend courses during the day, and then work for the university at night as a way of paying off their tuition. This night work included cleaning dormitories and class buildings, doing work on the grounds, preparing food, and so on. Incidentally, the vast majority of Hampton's first students attended on such an arrangement. So, yes, they were receiving some rudimentary instruction on the Three Rs, but the bulk of their "education" came in the form of unpaid labor around the campus (OK, technically they were paid, but their "salaries" were then given back to Hampton to pay for tuition).

One such person to attend Hampton according to such an arrangement was Booker T. Washington, who would go on to found Tuskegee Institute in 1881 according to the "Armstrong philosophy." Washington was a sort of disciple of Armstrong's; he admired his benefactor and sought to emulate his ways. As such, when he founded Tuskegee, he enlisted potential students in the actual building of the university itself, from laying the foundations, to making the bricks, to laying the bricks, and so on.

For much of the 19th century, and about two decades of the 20th, both Hampton and Tuskegee operated with this "Armstrong Model" of education. Both institutions definitely graduated a large number of teachers who returned to their communities and opened primary schools. Both institutions also definitely graduated successful businessmen who did not adhere to the social order that the Armstrong Model attempted to recreate. That said, a large number of Hampton and Tuskegee graduates did return to jobs as menial, low-paid labor. As such, the experience of a black student at a school such as Hampton or Tuskegee would be vastly different than the experience of a white student at a school such as Harvard, Yale, or even one of the land-grant schools founded in 1890. While "white" colleges/universities oftentimes sought to produce future leaders (whether they be political leaders from Harvard or agricultural mavens from Penn State), schools such as Hampton and Tuskegee sought to produce future followers.

Now, as to your second question, I can only say that "yes, other minorities did often attend different HBCUs, and often for the same sneaky reasons that blacks attended them." For example, in 1878 Hampton established a formal training program for any Native Americans that sought to attend the university. As you might expect, this program also leaned on the Armstrong Model.

SOURCES
Reconstruction: America's Unifinished Revolution by Eric Foner
Black Reconstruction by WEB DuBois (a former student not only of Harvard, but also Fisk University, another Black Ivy)
The Education of Blacks in the South, 1860-1935 by James Anderson (Anderson makes the strongest historical case regarding the so-called "Armstrong Model" that I've described above; he got his BA from Stillman, edit later a PhD from Illinois-Urbana)
The Education of the Negro in the American Social Order by Horace Mann Bond (a graduate and former president of Lincoln University)