The answer to this question, as you can expect, is going to vary from culture to culture. This answer will focus on England narrowly but on Europe broadly. England is particularly interesting as there are names derived from Old English practices and words, those that are Norse (usually Danish) in original form, and those those derived from the French, as well as names that are of Welsh, Scots/Gaelic, and Irish derivation--not to mention placenames that sometimes can be traced all the way back to their Roman roots.
So in England, as in Europe, surnames fall into several main groups:

1. Patronymics -- names usually based on a father's name, but occasionally on a mother's name or on another relationship (such as a marriage). (This latter category largely dies out by the late medieval/early modern era). This is where, in English, we get names like Johnson or Harrison. It's also where we get "fitz", as in Fitzsimmons (which would originally have meant "son of Simon'), which is from the Latin filius, meaning "son." Patronymics were pretty common in most of the Germanic-family languages and many of the Slavic ones, and are still the basis for surnames in Iceland and are still component of names in Russia (although now only as a middle name). Patronymics are sometimes "unmarked"--which would mean using a father's name without a possessive (e.g. the surname John rather than Johnson.)

2). Locative surnames, originally based on where you lived, which could be a placename or a geographical description. In England, this latter category is where we get names like Atwood (from atte wode, or "by the woods") . For the former, if someone lived in the town of Catterick, they might be known as "Robert de Catterick" from about the 11th-13th centuries; eventually the "de" drops off in the 13th-14th century period.
3). Occupational surnames, based on your job. In England, these would include names like Butcher, Shepherd, Tailor, etc.

4. Other descriptive bynames (sometimes called nicknames) - These could refer to personal characteristics or other descriptive words that might help to identify the person. In England, you get names like Black (which would probably refer to hair colour) or Berger (someone who is a burgher--a person dwelling in a town). You also get names based on personal characteristics (e.g. Chaffin - "moustache"), on animals (e.g. Bull or Stork) or even phrases (Goodyear).

So when do these names become hereditary? In England, for the nobility, the process is pretty much in place in the 11th century, particularly for placename-derived names. In London, the practice starts in the upper classes in the 12th century. It's a little harder to get a handle on for the lower classes, but we can start seeing entries in tax rolls in the 13th century where people have both a surname and an occupational name (think of Robert Catterick the tailor") or an occupation-derived surname plus an actual occupation that is different (e.g. Robert Shepherd the tailor). Per Reany and Wilson's Oxford Dictionary of English Surnames, "(Surnames) are well established in the south by the middle of the fourteenth (century). But there is clear proof that many men still had no surname and that many will still not hereditary."

Reany and Wilson's Oxford Dictionary of English Surnames is an indispensable resource for basic information on the evolution of English surnames. There are similar works for most other European languages/cultures.