As a very general rule, most surnames ultimately derive from one of the following:
Occupations, e.g. Archer, Bowman, Carpenter, Smith.
Geographical features or locations, e.g. London, York, Hill, Forest
Patronymics, e.g. Johnson, Robertson, MacDonald, O'Brien.
What might broadly be termed 'nicknames': Whitehead, Armstrong, Short.
A colour-based surname would likely derive from (2) or (4). It's easy to see how a person with unnaturally pale skin might be given the nickname 'White', likewise an unnaturally dark person might attract the name 'Brown' or 'Black'. When surnames were becoming established and fixed, some of these people would become known as 'John Brown' or 'John White'.
The surname Green is more likely to have a geographical basis. Again, it is easy to see how 'Paul From Green Hill' or 'Peter At The Green Forest' would eventually be reduced to the fixed surname of 'Peter Green'. (Or of course, possibly 'Peter Hill' or 'Peter Forest').
In this context, it is also clear why colours like 'Blue' or 'Purple' or 'Yellow' have not become common established surnames. A person's physical features are not likely to be blue or purple, for example. While the sky is blue in the natural environment, this is the same for everybody, so would not serve as a distinguishing feature when identifying an individual.
Really interesting, thank you!'