I think you need to look at them individually. For instance "Alison" is a French diminutive of "Alice", while "Madison" for women is apparently from a film character who chose the name from a street sign.

In addition to "Alison" and "Madison", which /u/ctesibius has already mentioned, there's also "Addison" and "Emerson", both of which became primarily female names in the last 20 years. The former was probably popularized by Dr Addison Shephard from Grey's Anatomy, but I can't think of an obvious reason for the latter. It's also worth noting that up to the 1930s, Alison also used to be primarily male (in addition to being a diminuitive of Alice, it can also mean Alice's son).

In fact there is a long history of giving girls traditionally male names, though the opposite almost never happens. As a result, many names that used to be exclusively male (Beverly and Hilary are other examples) have since become almost exclusively female, going through a brief unisex period. For more examples, see Unisex Baby Names: Names that morphed from blue to pink.