The amount of guards a medieval nobleman would have would probably depend on a number of factors. This could include such things as, but not limited to, their ability to pay them, the size of the castle in question and whether or not there were extenuating circumstances (i.e. war, civil unrest, famine).

Ideally, you would want the castle garrisoned by as few men as reasonably possible, even during a protracted siege as too many men would be a drain on the castles supplies. A well designed castle was typically designed to be able to be effectively garrisoned by as few a men as possible.

The next part is more related to the context of England in the High- Late Medieval period: It was probably fairly common for nobles to have a number of retainers, although this was not exclusive to knights, especially in the high and late Medieval period where we see Men-at-Arms who are not knights.

Of course the size of this retinue depended on what time we are talking about and the treasury and income of the noble in question. If a lord received income from many of his vassals instead of military service, then he might use that to pay for retainers instead. Obviously, powerful lords having a large retinue of men loyal to them would pose a problem for the crown as time went on.

We also have examples watchmen type groups being established to keep the peace, such as those established in the Assize of Arms and several instances where the Crown established a sort of Parliamentary or household guard (i.e. the Yeoman Warders)

Hope this helps.