Castles were placed in defensible positions such as elevated ground, often surrounded by water. These are for simple reasons, it's harder to fight uphill and it's harder to get over a wall when there's water on your side. The area of this I know the most about is the middle ages in what is now France.

I think your real question is "why did medieval armies bother attacking castles when they could just go around them?" Well, if you didn't take a castle you wouldn't gain anything. This is the heart of the feudal system. At the time, nations as we know them today did not exist. Power was held by regional lords who owned the town and the lands surrounding their castle (their fief). These weren't all independent lords, often a lord would have vassals who have paid homage to them, pledging them their loyalty (and various services, primarily military) and in return having control of a fief, which is under the protection of the lord. This could even go down more than one level, a king would certainly have many lords as his vassals who would each be powerful in their own right, some even being capable of providing hundreds of knights in service of the king. However, strong centralized power under the monarchy did not exist at this point. There was plenty of fighting between lords that wasn't a full-scale war as we know it today. So you can see that castles were not placed as a sort of line of defense to prevent enemies from encroaching further into your territory, castles were the center of the land owned by a lord. You couldn't just go to the next town to avoid the hassle because that town has a castle, too. The point of besieging a castle was to defeat the lord of the surrounding land and take control of it. Land was power, and taking another lord's land was often how you would expand your own territory.