The Roman deserters, about 900 in number, despairing of their lives, betook themselves to the temple of Æsculapius with Hasdrubal and his wife and their two boys. Here they might have defended themselves a long time although they were few in number, on account of the height and rocky nature of the place, which in time of peace was reached by an ascent of sixty steps. But, finally, overcome by hunger, want of sleep, fear, toil, and approaching dissolution, they abandoned the enclosures of the temple and fled to the shrine and roof.

That's all Appian writes (translated by H. White, source).

Neither Polybius (book 38/39) nor the excerpt (pariocha) of Livy's book 51 make mention of deserters.