You more or less have the idea down pat. Castles were a fortified place in a region of countryside which was protected from the enemy and that means as long as the castle stands the enemy can still resist with a safe base of operation. It's impossible (well not "impossible" but you get the idea) to grab hold of just the countryside and let them hole up in the castle and to go about your business in the war; if you proceed you have a significant amount of enemy forces holed up in that fortified city who can come out at any time and strike you from the rear. The only way to effectively neutralize the threat therefore is to surround the fortification and wait them out.

So castles became an incredibly useful tool throughout the Medieval and Early-Modern periods; no one really wanted to engage in battle as it was a high risk gamble. Raiding and prodding was the preferred 'safer' form of warfare; it's why you see all these enormous wars like the 100 Years' War and the 80 Years' War and so forth...it's not like they were just in constant engagement but the exact opposite; people didn't want to engage.

Castles were an extension of that; they protected the countryside from raiding as any army entering that region would be under constant threat of men from that position. However armies also didn't want to siege a castle as it was also a gambit except instead of it being a pitched field of battle it was gambling that they could outlast those inside. It seems almost like common sense that those on the outside are at an advantage but, in fact, are at a distinct disadvantage to those inside. Those inside had a significant store of food, were safe from the enemy, had shelter and fire and generally an established system of governance. Those sitting outside were completely exposed to the elements with little to no shelter; whatever they have is hastily put together and very temporary. They have a certain amount of food and can even bring food in but, regardless, supply lines can be raided. Generally they'd like to siphon food from the surrounding area but that includes fighting against a hostile population and just further attrits your men. Stagnation leads to starvation, disease, and just general desertion and attrition; all while the besieged are sitting comfortably in stone/wooden houses with beds and blankets and stores of fresh food.

An interesting case that shows the benefits and drawbacks of this is not as one would think from the Medieval period but the Napoleonic Wars, the First Coalition War for that matter. To put it briefly Napoleon, after defeating the Austrians initially, was placed into an enormous bind: He had to deal with the fact that a couple ten thousand men were holed up in Mantua but also had to continue his campaign against the Austrians who were recently defeated at Lodi. Mantua would be the center of the remainder of the campaign really; it could not be ignored and it drew Napoleon's forces thin. He constantly had to return from successful endeavors because the siege was threatened and if it was broken that's thousands of Austrians streaming out in his rear. What won it in the end, ironically, was help arriving. By sending in a couple ten thousand extra soldiers sent as "relief" and then re-engaging the siege it bursted the food capacity at the seams and sent the city into disease and starvation and general unrest. However Mantua could not be ignored because it did not matter how many battles Napoleon won; if the Austrians still had a large force inside of that city he could not reasonably proceed much deeper as it will always be a threat to his rear.

In Systeme Militaire de la Prusse Mirabeau would state:

without the help of the fortresses which studded the Netherlands, Louis XIV would never have escaped from the consequences of all the mistakes which his age and weakness led him to commit . . . After seven years of endless blunders and misfortune this triple line of fortresses remained unpierced; it exhausted the patience and resources of the enemy, and by checking them at this final barrier it earned the monarchy . . . a more tolerable peace than you would have expected.

and he was completely correct; fortresses saved Louis and it saved many before and after him. Marquis de Vauban would even claim in Traite des Sieges in 1704 that the Netherlands, with its ring of fortresses, made it more difficult to conquer than all of Asia. It would be those fortresses in Northern France which, in 1792-1795, would save France yet again from Austrian incursion as they were held up trying to siege every single one before advancing deeper into the heart of France. It allowed the Austrians in Northern Italy to hold onto a populace who did not want them there until 1866 because as long as their four fortresses remained what could the people do? Further fortresses would generally be strategically placed near mountain passes or other natural barriers which meant that if you wanted to incur into a foreign territory you had to take it! To put it another more political way; if one controls the entrance and exits to a region via strong fortifications what other option do the people inside have other than to be subject?

If you need any other proof just ask Napoleon himself:

In the last century [he wrote] there were people who asked whether fortifications were of any use, and there were even some sovereigns who concluded that they served no purpose, and proceeded to dismantle their strongholds. As for me, I would prefer to turn the question around and ask whether it is possible to concert a war without fortresses, and to this my answer is 'no'. Without depot fortresses we are unable to work out good plans of campaign, and without field fortresses (by which I mean posts that are proof against hussars and raiding parties) we cannot wage offensive war. Hence those generals who, in their wisdom, have rejected fortresses, are the very ones who are driven to the conclusion that one cannot wage an offensive war.

I hope that answers your questions! A great source on this, one of my principal ones, is Fire and Stone: The Science of Fortress Warfare 1660-1860 by Christopher Duffy. However most of this knowledge is gleaned from just general knowledge of the periods military history; another great source would be works like The Rise of Modern Warfare: 1618-1815 by H.W. Koch.