As described in 12 Years a Slave, in the south, they had crops all year round, so there was no time off for slaves. Sometimes slaves did more skilled labor when one couldn't work in the fields--they might be hired out as carpenters, carry heavy things, or clear new ground for cultivation. House slaves had lots of work all year round, cleaning, cooking, and taking care of children. Plus there is always work to do on a farm--taking care of animals and growing subsistence crops, making clothes, etc. for slaves with the misfortune to be in houses of prostitution, there was always work.

In the north, however, there could be no growing during the winter, so slaves might have no clear work in winter. It was expensive for northern planters to maintain a slave workforce in winter when little profit-generating work could be done. This is one reason why slavery ended up being less popular in the north, and was ultimately abolished in northern states.