The first serious emigration from what you describe as the colonies to the UK took place post American War of Independence with American loyalists arriving in the UK. Subsequent to that immigration to the UK from the colonies was mostly that of seamen, mostly from the sub continent, "paying off" ships in the UK and, sometimes, settling in whichever port they were in. The main centres of this pattern of immigration were Cardiff, Liverpool, London and the Tyne, especially South Shields. This book has some details Coolies, Capital and Colonialism: Studies in Indian Labour History. Rana P. Behal, Marcel van der Linden, published at Cambridge. It is a bit polemical at times, but the essential story is correct. More here http://www.southampton.ac.uk/archaeology/research/projects/lascar_lives_and_the_east_india_company.page also here The Invisible Empire: White Discourse, Tolerance and Belonging Again, this is a bit polemical with condemnation of the living conditions of lascars, whilst seemingly unconcerned, or unaware, that living conditions for British or other European seamen were pretty much the same.