No. It's something that quickly became apparent during the Hundred Years' War: small castles, fortified houses and small fortified towns often would not have enough men standing effective, constant watch for days and nights on end. As a result, they could sometimes be surprised with ladders up against the wall at night in the dark, allowing enough of the enemy to get a foothold to allow yet more to climb up, and the place would be taken. Bigger towns with higher walls and a bigger population were better able to resist.

Jonathan Sumption: The Hundred Years War