This isn't my area of expertise but I read a couple of books about it a few years back.

The troops at Rourke's Drift were able to defend themselves successfully in large part because they had fortifications. Comparisons are often made with the earlier battle of Blood River in 1838. There Boers armed with muzzle loaders and some artillery in a Wagonlaager (basically a circle of wagons serving as a fortfication) was able to defeat a numerically superior Zulu force. Fortification particularly against a foe who has to get in close to decisively fight you is a massive advantage.

I suspect the firepower advantage was also pretty huge. At the Battle of Isandlwana British rifle fire was able to stop frontal attacks for an extended period of time. Brave as a Zulus were they couldn't endlessly run into a continual gunfire forever.

Returning to fortification it perhaps shows how crucial this was considered to be in that Isandlwana involved little/no British fortification and was a defeat. And much of the controversy/what ifs tend to centre around the British troops being too widely deployed and the wagons not being circled. Although again returing to firepower, the other theory is (was?) that the British at Isandlwana ran out of ammunition at a key point and hence could not long pin down the Zulu forces.

As to why the Zulus at Rourke's Drift withdrew, from memory part of it was the Zulu force wasn't supposed to be there in the first place - they were meant to be somewhere else/doing something else and fell on Rourke's Drift as easy pickings. Also I'm reasonably sure Ceteswayo (the Zulu king) had ordered that fortified positions were not to be attacked. So they probably had to get going.

Their heavy loses and seeming inability to crack Rourke's Drift was also a likely factor. Always keep in mind on the battlefield information in imperfect - the attackers probably had no idea whether reinforcements were on the way or how close Rourke's Drift may or may not have been to breaking.

References:

• Zulu Rising by Ian Knight

• Rourke's Drift by Michael Glover