Doh, slave's Sundays not slaves Sunday's.
I didn't feel prepared to fully comment on this subject so I was going to abstain, but it seems like a lot of people are hoping for a response here. I'll try to get the ball rolling but I'd appreciate some help from the other brilliant minds around here.
First thing to keep in mind is that there was no singular slave experience, both because slavery existed in North America for centuries and because the South was not homogenous. A slave's experience in Virginia is not the same as in Mississippi is not the same as Kentucky is not the same as South Carolina. Further, experiences could vary immensely depending on the master, not that slave was ever a happy status to have.
Often times a slave's free time could be extensive or not. He/she might have to work any time the weather permits. They might have Sundays off or they might just get some time off at Christmas or whatever. Further, some slaves might operate under a labor system known as tasking, where they can stop as soon as certain quotas for the day are filled, or they might have to work certain hours no matter what, which typically meant as long as there is daylight.
Once a slave wasn't working for the master, they often used the remainder of the day to work for themselves. Taking care of cooking, keeping their domicile, caring for children, growing their own food, and doing the other odds and ends associated with life. Slaves didn't exactly have servants to take care of all that stuff.
Now, on to travel and mobility. Especially as the antebellum era wore on, restricting slave mobility became a big deal in many parts of the South, particularly in the Upper South where the risk of escape northward was greater. However, slaves were exceedingly skilled at remaining mobile anyway. Since slave families were often split up but not always separated by impossible distances, mothers or fathers would often sneak away to spend time with loved ones.
Further, escape was a regular and significant occurrence for slaves, but not in the way you think. Slaves tried to escape to the North very seldom compared with the number who escaped temporarily as a form of resistance. Master treats a slave poorly and that slave runs off and hides in the woods for a few days. Sure, the risk of getting caught was serious but it could also force the master to negotiate with slaves to an extent, helping encourage better treatment and reducing the severity and threat of punishment.
As for leisure activities when they did take place (such as on Christmas), it's a lot of the stuff you might expect. Playing games or friendly competitions, sometimes drinking if the master allowed it.
I've read two primary sources and one secondary source--written by Thomas L. Johnston, James L. Smith, and Nina L. Robinson respectively--from slaves who lived in Virginia and Tennessee from the 1830's up through the time of the Civil War.
The primary sources cover Johnson and Smith who were male slaves living in Virginia that cover the time period from the 1830's to the end of the Civil War. Smith escaped to the North in 1838 and lived as a freeman while Johnson, younger in age, lived in Virginia as a slave to the end of the Civil War.
Johnson, as an adult, would have been considered a plantation slave because he possessed no particular trade. Smith, on the other hand, was taught to be a cobbler as a teenager, and was running his own shop by himself, working about five miles from his masters plantation, when he was a young man.
Because of Smith's skill and larger distance from his master, he enjoyed more freedoms than the plantation slaves. Smith writes:
The other slaves were obliged to be on the plantation when the horn blew, at daybreak, but sometimes I did not get home till twelve o'clock; sometimes it would be night, and I always escaped a whipping.
Meanwhile, Johnson writes about needing to obtain a pass from his master in order to attend religious gatherings at night. And again this was due to his enhanced stature in having a speciality trade. But with all of these benefits in liberty, Smith still wasn't able to keep any money he made--beyond a meager portion--and writes that every couple of days or weeks his master would come to his shop and take all of the money he had made.
The common thread between both these men is they would attend religious gatherings at night. Johnston wrote about attending secret meetings with other slaves and holding religious prayers. And this was also corroborated by Smith who told a particularly interesting experience when he wrote:
I remember in one instance that having quit work about sundown on a Saturday evening, I prepared to go ten miles to hold a prayer meeting at Sister Gould's. Quite a number assembled in the little cabin, and we continued to sing and pray till daybreak, when it broke. All went to their homes, and I got about an hour's rest while Sister Gould was preparing breakfast. Having partaken of the meal, she, her daughter and myself set out to hold another meeting two miles further; this lasted till about five o'clock, when we returned. Then I had to walk back ten miles to my home, making in all twenty-four miles that day. How I ever did it, lame as I was, I cannot tell, but I was so zealous in the work that I did not mind going any distance to attend a prayer meeting. I actually walked a greater part of the distance fast asleep; I knew the road pretty well.
Smith was the more industrious of the two because he had more opportunities as compared to Johnson. A telling sign is that while Johnson was only attending religious gatherings, Smith with his force of personality was organizing and leading them. Because of Smith's freedom and liberty as a young adult, he was able to escape and make a living as a cobbler in the North when he made his escape. Johnson, who was barely literate, simply didn't have the same advantages Smith did. In reading their books, Johnson comes across as a gentle soul; whereas Smith seems more bellicose in nature. He once got into a fist fight with the master's son, but only received a minor whipping for it.
The third account I read told the story about a woman named Aunt Dice who was, in essence, the overseer for her plantation in Tennessee. She enjoyed more influence with her owners, but to answer your question specifically, in her account, it details how the slaves would have a big feast/party called "Cornshucking" that was held at the end of the harvest on her plantation. Robinson writes:
Merry cornshucking of the ante-bellum South, when negroes held high carnival amid swinging ears of corn and around the laden table of the harvest feast; when master and mistress bowed cheerfully to the grotesque rule of the merrymakers for a season—the swift-winged hours of the cornshucking night.
This party, in its simplest terms, was a way to get work done by getting all the corn shucked after the harvest. They would not only have the existing slaves on the plantation working, but they would invite slaves from other neighboring plantations to shuck the corn:
Invitations went flying across the country, up and down the river, to the colored acquaintances of neighboring plantations.
They were enticed by the plantation owners providing tables full of meat where they would you would find, "Pigs, lambs, and a tender calf [lay] slaughtered, and roasting slowly over hot coals in the trenches." In addition to this, the desserts were sumptuous as there was an assortment of "Jellies, custards, and dainty furnishings" for the slaves to eat as they labored shucking corn, listening to music, and singing songs into the night. Alcohol, in the form of rum, was provided to the slaves during the annual cornshucking.
The slaves would attend this party and stay up all night, but at the break of dawn as they made their way back to their plantations, many of them would be subject to work the next day. Robinson writes that at the end of the party:
The galloping horses churning the river, the swish of canoes, the soft stroke of paddles, the shouts and calls, proclaimed the hour of dawn and the departure of the guests.
With the sunrise Aunt Dice stood at her post by the rum barrel and kindly greeted the advancing row of laborers.
To wrap this all up, much was dependent on where the slave was living and the disposition of their master. Their were millions of slaves so it's hard to paint a complete picture without looking at a couple of different accounts, and I attempted to do that here. I didn't include any of the terrible stuff that happened in these stories, but there were plenty of instances of whippings, beatings, and lack of medical care that I didn't cover in here because OP didn't ask about it.
I didn't read any on the deep south, and I'd recommend looking through the website I've linked to in my sources--they are hundreds of primary sources to go through.
Sources:
I read 12 Years a Slave and also read the research on it by an historian who made the book her life's work. Here's what she (Dr. Sue Eakin) has to say about the specific area where Solomon Northup was enslaved: In Louisiana, by law, slave owners could not employ their slaves as slaves on Sundays. That is, any work done on Sundays by law had to be paid labor. If an owner needed the slaves to work on Sunday, then he had to pay them. And slaves could hire themselves out on Sundays to do other work. They used the money earned to buy themselves necessities and inexpensive luxuries. For example, clothing, blankets, tobacco, and food that they were unable to make themselves. Northup implies that everyone worked on Sundays so they could get funds for these purposes. They typically did manual labor, though Northup, being able to play the violin, was frequently hired out to perform at parties. They also worked at night (for no pay) tending areas where they grew their own food--they were provided with some food, but not enough to survive on. So they had to grow veggies for themselves and could sometimes tend animals, too.
They could not freely roam--they needed a paper from their master or overseer giving them permission to go where they planned to go. Any white person could stop a slave at any time and demand to see their papers. IF they didn't the right papers, they could be whipped and there would be a generous finders fee for the person who brought the escaped slave back. For that reason, Northup says it was common to have your papers intricately examined, especially by the poorer folk. The wealther folk tended not to bother. There were also nightly patrols on horseback (paid for by planters), searching for any slaves that might happen to run away or even just not be where he was supposed to be. One of the slaves on Northup's plantation (Wiley) had a wife on another plantation, and visited her one night. He was coming back in the early dawn hours and was caught and attacked by the patrol and set upon by their dogs. Then he was whipped. Then he was taken back to his master and whipped again. On the other hand, if a slave did have a note from their master, they could pretty much go anywhere allowed by their pass. If it was a couple of days' travel, they could go to any plantation, present their pass, and be fed and housed for the night free of charge
Besides Sundays, slaves also had a few days off at Christmastime--3 to 6 days. They could also work for pay during those days off, and it's implied that all the slaves did work those days because they needed the money.