Perhaps the main source for the abrupt departure of a load of Anglo-Saxons and Anglo-Danes in the direction of Constantinople is, as far as I'm aware, Orderic Vitalis, who also mentions that others fled into geographically more vague exile elsewhere, too. It seems to have a more haphazard, desperate quality the way Orderic describes it, rather than a grand orchestrated exodus. Having said that, it was a sufficiently long time after the Norman Conquest that there was plenty of time to plan it. Orderic doesn't give any numbers for those who went to Constantinople; he just says 'some of them'.

The so-called Saga of Edward the Confessor mentions three earls and eight barons leading 350 ships into the Mediterranean and thence on to Byzantium. Meanwhile, another 'Latin chronicle' describes 235 ships as you mentioned, but arriving in Constantinople in 1075 and resulting in 4,350 escapees from England settling in the empire. I haven’t read these two sources myself so I’m relying on D.M. Nicol’s reading of them, and he/she doesn’t pass much judgement on their provenance. That’s particularly awkward in the case of what Nicol describes only as a ‘Latin chronicle’ and I’m not inclined to lend too much weight to these figures.

Though the details are murky, it seems probable that some form of large group flight to Byzantium did occur at some point in the years following the Norman Conquest, as the Normans consolidated their power, cemented their hold on England and demonstrated that they weren't going to be easily shaken off. It also seems to be the case that, whether this group intentionally headed for Byzantium (as in Orderic's account) or drifted there over time (as in the Saga), they did end up bringing about a large English presence in the Varangian Guard. Chronicler Geoffrey Malaterra mentions 'the English called Varangians' heading into battle as the vanguard of the Byzantine forces, for instance. There's also a chrysobull of Nicephorus III from 1080 which mentions 'Inglinoi' alongside 'Varangoi' and Sigfus Blondal argues that this is 'irrefutable' evidence of an English presence in the Byzantine forces.

'English' is a little problematic in many of the sources, since it's not clear how many were Anglo-Saxons and how many Anglo-Danes, plus the references in Byzantine primary sources such as Anna Comnena aren't always clear about where exactly 'English' Varangians come from. Dawkins observes that western and north-western Europe was all pretty much a vague mass of indistinguishable barbarism to the Byzantines, and 'English' from a Byzantine source could just as easily mean Norse, Swedish or even Norman. Anna Comnena describes these 'English' Varangians as originating in Thule, a faintly mythic term which, Dawkins remarks, is in Comnena's case 'her general word for all the countries bordering on the North Sea'. It's a little (and amusingly) like the old saying 'it's all Greek to me' - the various languages of these miscellaneous barbarians are all English to me!

It's worth mentioning as an aside here that there's a bit of fudging around the genesis of the English presence in Byzantium. Nicol discusses the suggestion that the origin of Constantinople itself included 30,000 Britons taken east by the emperor Constantine after his coronation at York, and points out that references to the 'English' in some Byzantines could conceivably be referring to the descendants of these original settlers rather than to later Varangian recruits.

I don't know of any evidence for further migration from England to Byzantium on this scale. There has been some speculation that the latter days of the Varangian Guard may have seen continued recruitment from England but I don't know of any notable sources for that, and even if true it seems improbable that it would occur on the scale of 4,000 people at a time.

Sources:

Blondal (Benedikz ed. & trans.), The Varangians of Byzantium.

Chibnall (ed.), The Ecclesiastical History of Orderic Vitalis.

Dawes (ed. & trans.), The Alexiad of the Princess Anna Comnena

Dawkins, 'The Later History of the Varangian Guard: Some Notes', The Journal of Roman Studies 37.

Loud (ed. & trans.), The Deeds of Count Roger of Calabria and Sicily and of Duke Robert Guiscard his brother - by Geoffrey Malaterra.

Nicol, 'Byzantium and England', Balkan Studies 15/2.