A large part of the answer here has to be 'it didn't - Robin Fleming's 2010 book 'Britain After Rome' does a good job of setting out the evidence, essentially using archaeological material to argue that there was an awful lot more continuity than previously assumed, and evidence of wealthy, sophisticated societies - see the Staffordshire Hoard or indeed Sutton Hoo. It is true that trade with the Mediterranean declined - except at certain places, and Tintagel in Cornwall deserves a mention here - but trade with Scandinavia and the Germanic world picked up considerably, and there is a debate to be had as to how far this trade was ever important to most people (as opposed to the most archaeologically-visible people) in the first place. See my first comment below on sources.

One of the defining and unambiguous features of the end of Roman rule, however, was the decline of Latin literacy - and therefore the loss of sources that survive to give historians primary evidence for the period.

Our main textual sources are usually much later and all have their own agenda. Just about the only contemporary source with the Roman withdrawal is the Gemitus Britannorum - the 'Groans of the Britons' - written to the Roman general Flavius Aetius to convince him to bring his army to 'save' Britain. There are some transmission issues with this - it is preserved in Gildas and Bede (of which more later) - but we are left with a dramatic picture - 'the barbarians drive us to the sea, the sea drives us to the barbarians; between these two means of death, we are either killed or drowned.'

The problem is that this is exactly what you would write in a letter to persuade someone to bring his army to your aid, and that this is almost certainly coming from an upper-crust, highly-Romanised part of society - we have no reason to believe that it represented a uniform picture across geographical areas or the opinions of all people. In fact, one of the defining characteristics of the Roman state was its tax collections, and you can easily see how ordinary people might have welcomed its overthrow. Indeed, the Gemitus uses linguistic features - such as a neuter noun, mare ('sea'), which we know to have been absent in conversational Latin by this time, and which therefore suggest an author educated on Classical texts.

Turning now to these major sources - Gildas was a British Christian cleric writing (probably) about AD 500 (see Guy Halsall's 2013 'Worlds of Arthur', p54, for the outline of the debate) and essentially bemoaning the religious changes in Britain - new rulers had come in, no longer favoured the (Roman) Christian church, and Gildas was not happy with this. He makes the case that Britain has been ruined (indeed, his title is On the Ruin and Conquest of Britain) by these rulers, and that the abandonment of God goes hand in hand with the decline of the material world. Again - of course he thinks Roman rule was better; Roman rule meant that his religion, and people like him, were on top. His writing is hugely (Latin) literate, steeped in Roman rhetorical traditions, and generally proof that he belonged in the common elite culture stretching from Gaul to Egypt, from which Britain was now somewhat cut off. In many ways, Gildas had more in common with men of his class and profession in Roman Italy than he did with most people in Britain.

So too with Bede - his historical project in his Ecclesiastical History of the English People (again, the title is revealing) is to document the glorious re-conversion of Britain to Christianity, writing from about 200 years after the death of Gildas. It has to be a story of progress - Christian Britain has to be great, and therefore de-Christianised Britain has to be terrible. Bede's bias is well documented - we know for instance that he glosses over the well-known vices of the Archbishop Theodore of Canterbury (recounted in the Vita Sancti Wilfrithi by the less scrupulous Stephen of Ripon about the same time as Bede), probably because he did not think it right to disparage Church figures. We can also see his focus clearly in the fact that nearly everyone he writes about is either a churchman or a king. More severely, Bede was writing over two centuries after the fact - he is not in any sense a primary source, and what he recounts should be considered eighth-century mythology as much as fifth-century history.

In summary - the perception of decline post the Roman withdrawal from Britain is mostly an artefact of the nature of the sources and the historical agenda of those who wrote about it in the Early Middle Ages. Modern scholarship, placing more emphasis on the archaeology and working around these sources, is starting to change this perception. One key message is that things were not always the same across Britain, and we should not look - as Bede and Gildas did - for simple narratives that will cover the whole place and the whole period.

Sources

I have mentioned 'Britain After Rome' - it is worth a look at the review here to get a greater sense of the complexity of the picture; if Fleming comes down a little hard on the side of everything being rosy, Shippey here comes down a little hard on the side of everything being terrible - for instance, he uses the lack of roof-tiles, copper coins and hobnailed boots as a sign that Britain did indeed enter a 'dark age', but you don't have much need of roof-tiles unless you want to build a villa, coins unless you are doing long-distance trade, or hobnailed boots unless you want a standing army. So his picture of a 'bright age' (if that makes any sense) is predicated on looking like the Roman Empire - and it is not good historical practice to equate any change with a decline.

Guy Halsall's 2013 book ostensibly on King Arthur (Worlds of Arthur) discusses this period from a similar standpoint to Fleming.

On the end of the Roman Empire more generally and what that actually meant - two great books are Halsall's Barbarian Migrations and the Roman West and Chris Wickham's Framing the Early Middle Ages - Wickham also now has the Inheritance of Rome on Europe 400-1000 AD; it is very well reviewed though I have not yet been able to read it.

I should for balance's sake name Peter Heather as a writer who pushes the 'doom and gloom' impression of the end of the Roman Empire, together with Bryan Ward-Perkins (continuing the theme of revealing titles, his is 'The Fall of Rome and the End of Civilization') - but see Halsall's comments on his blog, e.g. here.