Fortunately, this is something John Laband (notably in Rope of Sand) has unpacked in some useful ways using both the James Stuart Archive, textual sources, and his own interviews. A lot of it turns on what he boils it all down to in his very brief treatment in the misleadingly-titled Zulu Warriors (because it's not all about amaZulu): the concern for being seen as cowards, unable to press the advantage, versus the tactical and strategic benefits you discuss. Some of this has to do with the ethos of the amabutho (age-regiments, though not always of regimental strength--sometimes more, sometimes less) within Zulu society, and their competitive nature as generational formations.

Four regiments were involved at Rorke's Drift, probably about 3500 men: the young iNdluyengwe (spot of leopard?) regiment of men around 27-28, with the senior uThulwana (butterfly/raising dust), iNdlondlo (viper), and uDloko (brawler) regiments, between 40 and 45 years of age. They were all eager to fight because they had been the unbloodied reserve force as Isandlwana, under the overall command of Cetshwayo's (the king's) recklessly aggressive brother Dabulamanzi. The balance against him was the strategically brilliant commander of the uDloko regiment, Zibhebhu, who was wounded in an earlier skirmish. So the brake on Dabulamanzi wasn't exactly there. Laband suggests it may have been a 'political wound' as Zibhebhu was not excited about such an invasion, Rope of Sand, 232; he also notes that many seasoned regiments turned back because Cetshwayo had not ordered an advance into Natal. The regiments who did move on from Isandlwana, however, were eager to engage in combat when the little store and hospital at Rorke's Drift--such a fixture that it was known as "kwaJim," or the country of Jim Rorke, whom older amaZulu respected greatly--sidetracked them from their path of raid into Natal. It was an ad hoc operation, not the linchpin of some grander strategy.

That arguably makes the decision to attack more confusing, but it may explain the retreat a little bit. The strategic stakes were low. The younger forces attacked first, the youngest iNdluyengwe first of all, with the elders joining in the two later attack waves; Zulu snipers (yes, snipers) and others with firearms tried to gain high ground to clear the area around the store and hospital, which is one reason why the defenders withdrew from the extended perimeterto the buildings. It seemed as though each wave was making some progress and each drew in more fighting men, but after the third attack failed to storm the store (but took the hospital), and burning structures gave the advantage to the defenders at night, matters settled down to misdirected chants and sounds to keep the garrison off guard. But the fight was over at that point, because Dabulamanzi had reports that British soldiers were on their way to reinforce the position from the flank (a report that was in fact correct, although the British believed the store had fallen already when seeing flames) and that meant the British were moving freely behind his forces. Although ill-tempered, he was not a fool, and he did not realize how little ammunition the garrison still had. By dawn, they had withdrawn (so no nodding respect scene with Michael Caine, sorry), with casualties somewhere around 15-20%, over twice the norm for a losing Zulu force. The experience had further soured amabutho on the idea of moving into more populated areas with similar strongholds. There was no glorious victory to be won in a siege, and the failure to take Rorke's Drift did in fact lead the returning amabutho to suffer taunts and accusations of being not men but women who ran away "for no reason at all" (quoted in Laband, Zulu Warriors, 235). Death was not preferable, but it may have been a close contest for some. However, being wiped out for no gain at all would have been far worse, and that is what Dabulamanzi feared even more once he knew more British units of unknown size and disposition were in the area.

When fuller news of the casualties returned, the jeering tourned to mourning and worry among the people who understood that this was just the beginning--the costs of Rorke's Drift were likely to be repeated. Cetshwayo suffered politically from this episode, both for failing to press the entire campaign into Natal but also for the failure of his brother. The damage was lessened by the fact that he had not ordered this attack, but increased by the fact that he had not committed more to an invasion of the colony. Cetshwayo, however, knew that the British would not negotiate after the embarrassment of Isandlwana, so it was more important to the king to be able to choose the place and time for engagements so as not to squander his forces.

[edit: Used a modern translation for two of the regiments. No wonder it made no sense! I got some rough ages for the regiments based on FWD Jackson's old three-part essay on the campaign ("Isandhlwana, 1879: The Sources Re-Examined") across Mar/Sep/Dec numbers in volume 43 of the Journal of the Society for Army Historical Research, 1965, but I'm sure there is a better source out there among the grognards.]