A medieval lord would build a castle in an area which he needed to exert control over. Oftentimes this would be a frontier of some sort, or on a strategically important bit of land.

In some cases, a village or town would spring up around/near the castle. The town often provided services and goods for the castle and it's inhabitants.

Castles in their grandest sense are not the only fortifications though. There are examples of walled towns (and cities), which existed to protect the town and it's people.

There are also country manor homes, which were somewhat fortified homes for the powerful or wealthy.

The residents of a castle varied. Some had a Lord of some sort in residence, others were overseen by men on behalf of the Lord. The garrison would live in the fortress, and I believe that at least some would have their families with them.

Generally a castle or fortification would only be abandoned when it was no longer needed. They require maintenance and upkeep. In cases where a fortress was not currently of military use (during times of peace, or if it was not along a hostile frontier) there would be a small staff that would maintain it and keep it useful in case it was needed.