The frequency of mistreatment is something I recently wrote about over here. The short version is that it's pretty horrific, though like anything individuals have different experiences and not everyone is uniformly cruel beyond the very extensive brute fact of slavery itself. Basically, owning slaves is inherently a form of cruelty at least as it was done in the Antebellum US. Beyond that, whipping, beating, destroying families through sale, and sexual violence are probably the most common forms of abuse but there are a lot of novel punishments that also come and go or are just worked into the everyday routine, like not feeding enslaved people enough to get through their day except at harvest time when you really need them at 100%.

As far as freedpeople returning to or remaining on their former plantations, that is something that happened but it's easy to misunderstand. People do differ; it's always possible to have a relatively benign former enslaver, or to expect one to respect the new legal rights one hopes will come into being. That said, the presence or absence of antebellum abuse probably isn't the key factor keeping those freedpeople who do remain close to where they were previously in that spot.

Few white Americans, North or South, were happy with the idea of black Americans freely moving about the former slave states. The accepted wisdom even by agencies that were supposed to (and often did) help the slaves like the Freedmen's Bureau is that the freedpeople need to be put back to work growing cotton ASAP. A lot of effort goes into getting them to sign coercive labor contracts with former enslavers which aren't quite slavery but can get pretty close. The former enslavers, naturally, are all for this. So it's not necessarily a case of wanting to remain, though it's certainly possible that some relatively less brutal enslavers might have owned people who stayed on under the presumption that the worst was over.

That may have been a bad gamble, but if your life has been one of frequent involuntary relocations and you now have a family and some stability in a local black community, the choice not to move away can be an important part of freedom too. No more selling away loved ones, or being sold away from them, means a lot. Replicating it voluntarily might happen too -many enslaved people put a lot of effort into getting in contact with lost family...which didn't go so well when someone had moved on and found a new spouse, for example- but it's a reason to stay or come back.

We also have to factor in that land reform never really comes to the American South, so even if one wants to keep growing cotton (unlikely, but maybe) the opportunity to do so on one's own land never arrives for a great many freedpeople. There aren't many places one can hope to go to. Some freedpeople get land anyway and a few more manage to get some kind of rights to a few years use of former plantation land, but federal and state-level policy reverses kick in fast and much of the land that either was available or was to be put up for auction at reasonable prices ends up back in the hands of the enslavers.

Sources

As linked for abuse suffered by slaves, and additionally Johnson's River of Dark Dreams on malnutrition of enslaved people. For Reconstruction, naturally, Foner's Reconstruction.