The fact of the matter is that they could do anything they wanted to. That's the awesome thing about being rich and not needing to have a job.

Okay, but of course "everything they want to" has limits on it, whether technological (can't play video games in the 1910s), legal, or social. "Social" is probably the most important thing here.

The hardest thing about trying to inhabit the period British aristocrat mindset - or at least, what seems to me to be the hardest thing, form the historical fiction I've consumed (and discussions of the same that I've seen) that misses the mark - is registering how crucial the issue of social acceptability was and how internalized it was. There were very many options for passing the time that the vast majority of the aristocracy and the landed gentry would simply not even consider. For instance, as I wrote in this very recent answer on New York high society, joining the entertainment world as an actor/performer could spell social ruin - you couldn't simply go off and be on the stage because you wanted to be, unless you were willing to lose most of your family and friends. You would most likely not even think, "oh, I wish I could go do that if only stinky boring people would stop telling me not to," it would simply not present itself as a valid line of activity.

Social acceptability was particularly an issue for women of this class. It was important to perform the required social roles, as a younger unmarried woman (to show your suitability for marriage), as a married woman (to preserve the family dignity and honor), and as an older unmarried woman (because you had no shield whatsoever and people were already looking at you as an oddity). "Accomplishments", as /u/ConcertinaTerpsichor says, included largely artistic pursuits, and were an important aspect of this gender performance, although it's important to bear in mind, because they're so often associated with the unmarried girl's experience particularly, that these were acquired through schooling in the teen years and then could be practiced ever after through adulthood. While it served a purpose to be seen embroidering, to have your art hung on the walls, to play the piano for guests, these were also forms of recreation, just as they are today.

Another important activity for upper-class women was charity work. In the eighteenth and early nineteenth centuries, this was likely to take the form of direct aid to destitute families in the local village: bringing over food from their own kitchens to supplement the meager provisions of a widow, or making baby clothes and diaper cloths for a tenant family with many children. By the late nineteenth and early twentieth centuries, it might be more likely to involve some sort of association or society. While the idea of women's place being in the "private" sphere is a common way of understanding gender roles in the period, charity and reform work allowed women to have a place in the public sphere that was seen as very well suited to their feminine, maternal natures: it spread the nurturing and improving influence they brought to the home out to the wider world. This is something we very rarely see in historical fiction, perhaps because it seems like a boring and conventional way of doing good.

They also needed to keep up their place in their social networks by handling correspondence and paying calls. Again, the former tends not to happen in historical fiction and the latter tends to be portrayed only as "young heroine forced to sit around and make meaningless small talk", but ... this is how people kept in touch with and met up with their friends and acquaintances. That could involve basic small talk about the weather, but it could also involve passing important information about politics, social movements, and family members, and everything in between. Between the length of letters and traveling around town to pay calls, this could take up quite a bit of time.

Again as /u/ConcertinaTerpsichor mentions, shopping was another major activity for women of all ages. Cultivating an appearance that showed taste, refinement, and wealth was part of the required social performance. This meant patronizing the right shoemaker, the right dressmaker, the right milliner, etc. While ready-made clothes were pretty common by the time of Downton Abbey, wealthy women and those looking for a special piece (particularly for a wedding) still tended to go to custom makers. If it was within their means, they might go to Paris to patronize the couturiers like Doucer, Beer, and Worth; they might also go to London-based firms like Lucile or Redfern. In either case, that would require travel, which might also offer other occupation like visiting art exhibitions or going to the theater. (And just in general, the Edwardian elite did a lot of long-term traveling abroad.)

And then there are recreational activities that we would still recognize as recreational! Tennis and croquet were popular summer sports; riding horses, just around the countryside or as part of a hunt, was a required pastime; they would have all the time they wanted to read novels and non-fiction; visitors could stay with them for weeks, or they could go to stay with someone else for weeks; they had parlor, board, and card games; and, of course, people had hobbies to indulge.

The primary purpose of aristocratic women was to provide children of suitable lineage, and secondarily to enhance the prestige and the property of their partners.

So a young girl would spend her time being educated and being trained in suitable female accomplishments like needlework, music, watercolor, etc., as well as riding, or perhaps lawn tennis. As she approached marriageable age, she would learn dancing and acquire a wardrobe, for which there had to be much shopping and many fittings at the dressmaker’s. She would probably be a debutante and attend various balls in order to be checked out by eligible bachelors.

Once a young woman was married, she was supposed to fall pregnant fairly quickly, but until then she decorated the house, rode to hounds, built up her wardrobe, planned gardens, gave dinner parties, and attended social gatherings as a ornament to society and a credit to her husband. (Hopefully.)

Once the first child was born, their education and upbringing became a primary project, and so on for the rest of however many children came along.

Then came the huge hurdle of arranging suitable marriages for them. Partners of the correct status, age, wealth, suitable character, and suitable lineage had to be found, and l then competed for. Finding a fiancé/e could be very difficult if there wasn’t plenty of money or sufficient personal attraction.

Once grandchildren were born, the cycle of raising them to marry well began all over again, though at a remove.

Downtown Abbey makes their lives look very tranquil, and of course compared to most people’s they were. But within their own social sphere and by their own standards they had a lot to do, plan for, and worry about.

Notice that the plots and subplots really turn on who each daughter marries or doesn’t marry. If there were sons it would have been a totally different show. But for women of the time (and to some extent today) who you married determined EVERYTHING in your life — where you lived, what you ate, how you spent your time, how safe you were, etc. So while modern audiences can enjoy the romance, there is also a deadly serious undertone throughout the whole story.