I've never heard this. Do you have a source?

I would like to point out that the music referred to today as "Bluegrass" is a very particular style of music in the style of post-war recordings by the famous "Blue Grass Boys" (Earl Scruggs, Bill Munroe, Lester Flatt, etc). This music is heavily inspired by Appalachian American music, but isn't traditional by any stretch of the imagination. This difference is a widely agreed upon fact by performers in the genre, with many musicians referring to traditional Appalachian music as "Old-time" music, and the new-school as "Bluegrass."

One of the big differences in the two lies in the banjo technique. In Bluegrass music, the banjo is played with a complicated finger-picking style (Which was invented by Scruggs of the Blue Grass Boys) while traditionally the banjo has been played with a style known as "clawhammer" where the thumb and fingernails alternately strum the drone and fretted strings respectively.
This is the style originally invented by West Africans, brought to America by the slaves, and became standardized by white minstrel performers in the 1830's.

Now that that is out of the way, it's time to address the question, which I will take as the following: "Is traditional Appalachian music racist" for the reasons described above. And the answer is, it's complicated. Much of this has to do with the history of the minstrel show (which is an extensive subject many, many historians have wrestled with and much in depth work has been written about) which most certainly was racist in it's depictions of black characters (that were often whites in ridiculous makeup) and the music often followed the same theme.

However, the music did incorporate elements of North American Black music (fusing elements from Africa, the Caribbean, and Christian hymns) which became a hit with White Audiences, and led the Black elements to become an major part of American music in general, and still dominates to this day.

The minstrel songs in the traditional Appalachian repertoire were not in any way singular to it, they were simply the popular songs of the day, and many such as 'Turkey in The Straw', 'Hello My Baby' the song Michigan J. Frog sings, and anything by Stephen Foster are still popular.

By the turn of the 20th century, the lines became even more blurry as minstrel music written by Whites in imitation of Blacks found it's way into the repertoires of Black musicians (as seen in the 'clawhammer' clip above) as Black musicians began to become stars on their own through early ragtime, blues, and jazz.

In short, is Bluegrass racist? No more racist than any other popular music from about 1950. Was traditional Appalachian music racist? No more racist than any other popular music from about 1850.

I can provide sources for much of this information, but it's alot of sources and I really need to go to bed (its 1 am for me!) I've done alot of reading on the history of Jazz, Blues, Vaudeville, the banjo, and Appalachian music, and would be happy to provide specific sources if you ask in the comments.

The use of banjo predates bluegrass music significantly. So you're actually talking about a number of different components. Bluegrass music is not the same as minstrel shows. And the banjo was an extremely popular instrument used in multiple genres of music in the late 1800s/early 1900s - one of which was Minstrel shows (for a number of reasons that I could go into if people are interested). The banjo is definitely tied to instruments in Africa (a skin - typically a goat skin - is stretched over a gourd with a stick attached. A drone string is combined with longer strings, and the clawhammer style of playing is similar to the way many of those instruments are played in Africa. Typically the African instruments have three or four strings instead of five).

Bill Monroe is largely considered the father of bluegrass music. He was in a brother duet group with his brother, Charlie Monroe. Brother duet groups, such as the Monroe Brothers, Delmore Brothers and the Blue Sky Boys were extremely popular in the late 1930s and early 1940s (typically, one brother played the guitar and the other played the mandolin).

Bill Monroe grew up in the Kentucky area. He was influenced by a number of different genres. His Uncle Pen played fiddle in an old-time fiddle style that would have been greatly influenced by the migration of Scottish people in the Appalachian area.

Bill was also greatly influenced by Arnold Schultz, a black musician that played a thumb-style of guitar picking, similar to Merle Travis, Chet Atkins and Bill Broonzy. His style of playing incorporated what is now considered blues and country. Back then it was largely called "hillbilly" music.

Another huge influence on Bill Monroe would have been the Baptist church, and the gospel singing (which provided bluegrass' three and four part harmonies).

Similar to many forms of music in the United States, this amalgamation of multiple genres of music in one area created a form of music that is known as bluegrass (be aware that music was much more regionally influenced back then). The "back beat" of blues (which is different from the roots of country, where the beat is on the one and three) - the three part harmony, and the European folk music.

The last element that you're talking about is the banjo. Minstrel shows are not a part of the bluegrass banjo that you are talking about. Minstrel shows incorporated the banjo, vaudeville humour and African stereotypes as a form of entertainment - but the popularity of Minstrel shows peaked at the turn of the century. By the time bluegrass music was in it's early stages, Minstrel shows were largely non-existent.

The banjo music that is related to bluegrass was very regional to the North Carolina area (completely independent of Minstrel shows). The roots of that music relates to a style of banjo playing that was popular in the late 1800s and early 1900s called "classic banjo." (Here is Fred Van Eps playing Maple Leaf Rag in that style).

A two-finger style of banjo playing was very popular in the 1920s Here is some Doc Boggs as an example - and the North Carolina region had many banjo players playing in this style. This is different from the clawhammer style of playing

One of them was a guy named Snuffy Jenkins. Snuffy Jenkins (and another guy named Wade Mainer) had heard this classic banjo style, and started to incorporate a three-finger style of banjo playing. Here is Snuffy Jenkins playing Cumberland Gap

Snuffy Jenkins started to play with metal fingerpicks, which brought a very syncopated rhythm to playing. It's important to note that Don Reno and Earl Scruggs' family grew up very close to Snuffy Jenkins. Whether Earl learned his style from Snuffy is a matter of debate. Earls' brother, Junie, likely learned the three-finger style from Earl, and Earl learned aspects of the playing from Junie (and developed his own style from there). I'm not saying that Earl was not an innovate of the three-finger style of five-string banjo playing. But he was definitely not the first.

Lester Flatt, a guitar picker, had played with Earl Scruggs before. Lester was playing with Bill Monroe in 1945.

Now, banjo players typically played a comedic role. Stringbean was playing with Bill Monroe, but Bill Monroe was not happy with the clawhammer style of banjo playing that Stringbean provided (he wanted a style that provided more "drive" to the music). Lester Flatt recommended Earl Scruggs as a player. Interestingly, Don Reno was lined up as a banjo player in 1943 for Bill Monroe, but Don went to serve in the army.

Many people believe that bluegrass was born when Earl Scruggs joined Bill Monroe. Bill Monroe and the Blue Grass Boys would play on the Grande Ole Opry and would influence many others to play in that style of music (e.g. Stanley Brothers, Osborne Brothers, McCormick Brothers, etc).

Bluegrass standards were Baptist gospel songs and hymns, blues songs, country standards and European folk songs.

EDIT: I cleaned up some of the writing - and here is a significant source - Neil Rosenberg wrote what is largely considered the definitive book on the history of bluegrass music.