Just put some prisoners of war in a big cart with a roof and start mining through the stone and mortar. The materials for a siege tower would still be higher than just a low profile cart with a thick protected roof. The engineering knowledge and quality of materials for siege weapons like a catapult seems so expensive and unreliable. People didn't use battering rams when mining stone. Why didn't people use the tool made for breaking stone, a pickaxe?
At times they certainly did! The use of picks or axes against fortifications even shows up in some medieval artwork such as these examples from Manuscript Miniatures.
Konstantin Nossov in Ancient and Medieval Siege Weapons calls this tactic "direct" undermining, which is where soldiers would attack the base of the wall directly, usually under the protection of large shields or a mobile cart with a roof, and dig away until they had either created a breach or done enough damage to cause the wall above to collapse.
The problem was that even with pickaxes, digging through solid stone was still a very slow and tedious process, and the whole time any attackers standing directly at the base of the wall would be very vulnerable to iron darts, heavy stones, burning coals or pitch, etc. constantly being thrown down by the defenders above.
So, besiegers would typically try to look for an alternative. If an extremely heavy, iron-capped battering could be brought close enough to the wall then it might be able to cause a collapse more quickly than sappers working with pickaxes only. Ladders or siege towers could be used to attack the top of the wall and allow soldiers to go over it rather than through it. Trebuchets, catapults, or cannons could be used to cause damage to the walls from a reasonably safe distance. Or the attackers could undermine the walls by starting a tunnel and approaching the fortifications from underground. This last option would certainly take much longer than trying to dig through the wall directly, but meant that the sappers would be completely protected from missiles during their approach and it meant there was a chance for the defenders to be caught completely by surprise when it was finally time to collapse the wall.
Essentially what it boils down to is that even prior to the use of gunpowder weapons no wall on it's own, no matter how strong, was likely to be more than just a brief obstacle to a determined attacker. What made a medieval castle work were the efforts made by the garrison troops inside to actively defend it.