Small bag that hold things have existed forever, however other than money there is usually very little to carry around (people usually carry a basket if they are shopping for food). Prior to the late 1700's only a pocket either tied around the waist or sewn in the dress is necessary. In the late 18th and 19th century as regency dresses became more narrow so there is not a lot of room for pockets, women began carrying a small drawstring purse called reticule, which might be one of the earliest purse especially made for women.

In the mid-1800's middle class women began wearing this keychain like item called chatelaine that hooks many items they might need running their own household including keys, notebook, scissors, etc. It's a very iconic item of a middle class woman because an upper class women would not need to hold the keys and a lower class woman wouldn't be able to afford such an intricate accessory.

If I have to put a date on modern purse I would say 1910-1920's. Not only many women worked during World War I so they need to carry things, but the flimsier dresses cannot support the weight of pockets or chatelaine anymore. In addition in the 1920's wearing makeup because acceptable and they need a way to carry their makeup around. I have this compact in my collection, it has a chain to carry like a purse, with a smaller well on the left is for rouge, the larger well is for powder. The brass compartments on the right hold some coins. So it's a hybrid makeup compact and purse.