Before the Angles and Saxons came from the European continent, large parts of Britain (mainly present day England and Wales) was inhabited by Britons, a Celtic people. The original theory was that the Britons were quite literally pushed west by the invading Germanic tribes, to Wales as well as Cornwall in south-western England. A more modern theory claims that the Britons to an extent were assimilated by the Anglo-Saxons. Whatever the case, there is no doubt that the heritage of the Britons still lives on in Wales, Cornwall and, as you correctly assumed, Brittany in northern France.

Based on accounts from Gildas and Nennius, two British monks who lived three hundred years apart (6th and 9th century), and supported by archeological evidence, the Britons settled in present day Brittany in two waves. The first wave is stated to have consisted of Roman auxiliaries (i.e. soldiers who were recruited from various Roman provinces rather than Italy itself) who settled there after withdrawing from Britain in the late 4th century. The second wave is said to have followed the Anglo-Saxon invasion of Britain, when some Britons chose to escape across the sea rather than into Wales or south-western England.

No matter what really really happened, there is no real doubt among historians that the Bretons are descendants of the Britons.