Britain was Brythonic Celtic speaking at the time of the Roman invasion. Gaelic speakers came to dominate Ireland, and then Gaelic-speaking Irish raiders - or "Scots" - invaded western Caledonia after the Roman departure, leaving the Highlands and Islands Gaelic speaking. There is always the question of pockets of indigenous, non-Celtic speakers surviving in Britain at the time of the Roman invasion, but that is a matter of debate.

edit: for clarification - Welsh is Brythonic as was Cornish, which is enjoying a revival. The language of Brittany is also Brythonic, which is also called "P" Celtic, because a consonant shift caused a difference in the language of the "Q" Celts or Gaelic speakers of Ireland, Scotland, and the Isle of Mann (and formally of Galicia in Northern Spain).