A lot of depictions of this are like a meatgrinder, where soldiers climbing up the ladder practically walk into the enemy weapons. What kind of tactics were used to avoid this?
Perhaps one useful way to look at it would be to consider that castle defenders were often very low in terms of numbers. Perhaps this would be different in a large fortified town, but usually the defenders would be significantly outnumbered.
Which makes a certain kind of sense. If you are to resist a siege, you not only need combat power but also supplies (food, water etc.) Having a large armed garrison could just as equally be a detriment as a benefit, given that they would consume food and water that much faster. In fact during the siege of Chateau Gaillard in 1204, the French King refused to allow civilians through his lines from the castle. This was an attempt to deliberately force the garrison to use its supplies faster to feed the populace, rather than resist.
Overall the strategy would be to have as few defenders as possible defend against a much larger force, therefore tying up enemy forces for longer and lower cost. If the enemy got ladders to the walls, you may even be unable to defend every inch of the fortress - instead you focus on those areas under attack and hope the enemy doesn't have another attack coming on the other side of the castle!
In addition, and many others could probably add more, but it seems to me that siege commanders were far more inclined to force a breach in the walls (through mining, ramming or bombardment) than to attack the wall itself. If they did, it seems siege towers were the preferred method, and in my research I've come across few escalades. The siege of Jerusalem in 1099, for instance, primarily involved siege towers, despite the fact that ladders would likely have been easier to make due to the scarcity of wood.
The most popular tactic remained waiting the garrison out. In this way you minimise casualties and inherit an intact fortress to boot!
Carpenter, David (2003). The Struggle for Mastery: Britain, 1066–1284
Riley-Smith, Jonathan. The Crusades
McGlynn, Sean (1 June 1998), Useless Mouths
Bachrach, Bernard S (1994). "Medieval siege warfare: a reconnaissance". Journal of Military History