In Battle Cry of Freedom by McPherson, it is stated that "Not all the 'bummers' (foraging soldiers that roamed through the countryside) were Yankees, however. Georgia unionists and liberated slaves hung on the flanks and rear of the army and lost few chances to despoil their rebel neighbors and former masters."

"Southerners" and "En masse" need to be defined more readily. Southerners did join Sherman, although most of these were probably recently freed blacks due to the Emancipation Proclamation that followed their rescuers. Georgia unionists--I'll need to find more data about this but unionists weren't that common in the lower South. According to Battle Cry of Freedom the average vote in favor of secession in the lower South states (MS, AL, FL, GA, LA, TX) was around 80% and considered "a fair reflection of white opinion". So Georgia Unionists and especially freedmen may have joined him en masse. Southerners in general probably did not, though.

Verdict: Did Southerners join Sherman en masse? Depends on your definition of en masse. Yes for freedmen and Georgia Unionists, probably not for Southerners in general.