Quite an interesting question!

The Old Bailey records, which have been digitized and put online, are an excellent resource for this sort of thing, as handbags/reticules were often stolen or stolen from. We have the most detailed records from the early decades of Victoria's reign; in the second decade, the contents of handbags do not tend to be listed.

The most common things to appear in the court records as being inside reticules are handkerchiefs and purses - that is, coin purses. At this time, these might be the sort of leather pouch with a kiss-clasp that the term "coin purse"/"change purse" calls to mind, but also might be a knitted or crocheted long purse (also called a "miser's purse" in modern sources, but this seems to be a term invented in the 1910s, when use of this kind of purse was coming to an end; it's a cheeky reference to the number of steps it took to get coins out of it. Laura Camerlengo has a great paper on these purses here). These often turn up as the only contents of a stolen bag, and could be considered the "default" - what most women would be carrying around on an average day.

Small items of clothing also tend to turn up - most commonly gloves, as they might be taken off or put on during the day, but occasionally stockings or a collar. The latter would most likely be recently purchased goods than stuff that was wanted for quick access during the day, given the difficulty of replacing them in public.

We see a certain amount of jewelry being carried: rings, buckles, necklaces, brooches, bracelets. Small items for everyday use are also common, like eyeglasses, scissors, keys, calling-card cases, printed books, and notebooks.

Some interesting and detailed testimony: in May 1837, a woman testified that

my daughter said, "Give it him, mamma," or "Let him have it," and on that he took it from me—he had said, "Give it me," meaning the bag—he took it from me forcibly, and then ran off—it contained a pocket-book, a silver pencil-case, a pair of scissors, a pair of silver-mounted spectacles, a pocket-handkerchief, a gold watch-key, and a lace collar—I think I have seen the pencil-case since at Mr. Turner's, the pawnbroker's—there were several other trifling things in the bag which I cannot recollect ...

In August of 1843, George Newton stole a reticule from Lydia Linkson,

containing 2 sovereigns, a bunch of keys, a pair of spectacles, a banker's book, and some other articles

You may have come across a pop history piece about the "sexist history" of pockets on Racked back in September, or when it was rehashed and simplified on Mic.com afterward. It asserts that exterior pockets - handbags - were prevalent during this period due to social strictures aimed at preventing women from navigating public spaces, traveling alone, or carrying unsanctioned publications. This is pretty much bunk.