To add to u/olakreZ very straight to the point answer, though I too will not add any bibliography as my books are away from me at the moment.
It's not right to think of the castles as purely defensive fortifications. Castles served multiple roles at once and it's this combination of roles that led to their importance in medieval warfare.
For starters castle was where people lived, even outside the warfare context. It was practically a fortified house, sometimes could count as small town (but rarely, it was more common for town to form near or around castle). So the people inside castle had most likely heating, water, food stockpile and weren't just soldiers defending from a siege but people going on about their lives as normal even if siege was underway.
Since castle was usually home to a local lord, it often also housed his family and guards. That's quite a group of people who most likely were trained in combat. Turning your back on hostile castle was like turning your back on armed man - at any point castle inhabitants could open the gates and attack you, while you had no such option yourself.
This in turn created a third purpose of a castle - control. The fact that local lord can come and cut you down at any time, while retaliation against him will hit the hard stone wall, was reality of most his subjects. With castles lord could control local population even if they were of hostile culture or faith. Thus if you wanted safe passage for your supply lines during war, you had to make sure every castle along the way was either empty ruin or under your control.
In Short - think of a castle like an entrenched army with limited movement (like dog on a leash for example). It's never a good idea to ignore and turn your back on the enemy.