How many of them would live there and what were their living conditions like? Also, what would they do when their master was "out of town"?
To get an idea about how many people of what types might live and work in a large castle, let’s look at Hame Castle in Hameenlinna, Finland, about 60 miles north of Helsinki.
This was a major royal castle of the Swedish Crown. One of the three most important castles the Swedish Kings held in Finland.
It was built in the 14th century, but to understand the number of staff, we are going to look at the castle in the 16th century, because particularly good records for the time period 1539-1570 are available for this castle.
Brief wikipedia article on Hame castle with some photos: http://en.wikipedia.org/wiki/H%C3%A4me_Castle
This was a generally peaceful time in Finland, and Hame castle was no longer near any Swedish frontiers. The castle was a royal administration center in the period, more than a strategic fortress. (Though there were often soldiers mustered at the castle or training at or near the castle.)
Staff at the castle can be grouped into several categories:
Agriculture: The castle controlled three large landed estates, growing primarily rye, peas, beans, and raising cattle, sheep, pigs, and horses (a royal stud farm). Of course, most of the agricultural laborers lived on the estates, but some of the managers, those making agricultural tools and implements, those storing and transporting the produce, and several fishermen (from 1 to 7 at different times) lived in the castle.
About 10-20 people involved with managing storing or producing food lived in the castle.
Artisans and Domestic staff: In this category were artisans such as the blacksmith, carpenter, masons, weavers, and their staff, as well as the food service and domestic service staff.
About 10-20 Artisans and Domestic staff lived and worked full time in the castle. This number could grow considerably however when major work was being done on the castle or when the castle was crowded and more domestic staff were needed to look after extra numbers of officials or soldiers. For example, in 1559 work was being done to make the castle more defensible against cannon (castles were becoming obsolete due to cannon and star fort defenses were beginning to be built across Europe. In 1559 a gun tower for cannon was added to the castle and in this year 65 craftsmen from different trades were employed and living in the castle, and 47 people were employed in Agriculture management and Domestic Service to help look after them. The construction of star fort type defenses around the castle continued for a decade, and the number of craftsmen and extra workers in the castle remained high (though not as high as 1559). There were always a fluctuating number of temporary artisans living in the castle while engaged in some contract or other. There were also a fluctuating number of servants who lived outside the castle but would be engaged to work in the kitchen or housekeeping on a temporary basis when there was extra demand.
Officials, Soldiers, and their families, staff and servants: In charge of the castle was the Bailiff (usually a member of the minor Swedish nobility). His main assistants were the Scribe, who supervised an assistant scribe and a copyist, and the underbailiff (The scribe was paid more. The castle had a hard time keeping underbailiffs. There was a high turnover for some reason.) There were also a group of Squires, who made up the castle’s permanent garrison, and also served as foremen and delivered tax monies to Stockholm, as well as daily guard duties.
Generally, this category of officials and their families, and permanent garrison, consisted of 15-30 people. In addition, there were almost always some cavalry or infantry troops in the castle. These were not troops assigned to the castle, but the castle formed a recruiting, training and re-supply base for local companies and regiments so they often had some troops based at the castle for weeks or months at a time. There would usually be 50-100 soldiers staying in the castle.
Throughout these 31 years, at least 100 people lived and worked in Hame Castle. Frequently there would be up to 150 people based in the castle. During the decade of the 1560s there were generally 200-250 people in the castle, but this was because of extensive works to make the castle more defensible against cannon, and the generally higher number of soldiers making use of the castle for temporary periods in this decade.
http://elektra.helsinki.fi/vilksum.html
The inhabitants of the castle probably lived somewhat better than most. The Baliff's table served its diners an average of 4,000 calories per day, with meat (mostly beef) at most meals. The Servant's table served its diners 2,800 calories a day and about 3,300 calories a day on holidays, with little meat except on holidays, but quite frequent fish as well as grains and vegetables.
The inhabitants of the castle probably never saw their 'master', the King of Sweden during these 31 years, as he never came to the castle. The castle would have probably run much the same as always when the Baliff was absent, as his subordinates would have kept things going.
Related question: Is there actually any such thing as a "Gothic castle"? My understanding is that Gothic architecture relates almost exclusively to cathedrals, not castles.