A good starting point would be Mick Aston's books (you'll recognise him from Time Team), his writing on British landscape archaeology and towns is, in my opinion, second to none. Then I'd recommend anything by John Schofield, not just because he was once my tutor :)

If a town was planned like a fort then there would indeed be an overspill from the central, fortified area. Some settlements were planned that way - Vindolanda is a good example with its vicus on the approach road. Obviously the high-status/high-value buildings would remain in the most secure area.

In medieval times towns were not necessarily fortified with walls, many had town ditches with bars on the crossings into the town and chains across water approaches. These were a function of taxation as much as security, of course. With these towns, as with walled towns, the townspeople would own land beyond the boundary - this was usually the land used for farming, timber, liming, mining, all of the things that require specifically-located resources and which need a lot of room to process. Towns could also spring up around Monasteries and random port-landing sites. Sometimes a village would come into wealth through luck and circumstance and a town would spring up there. Not all towns were planned, they just... happened.

The growth (and shrinkage) of towns would, as you say, be hard to predict. I work particularly in Holderness and I always cite the example of Patrington which has an incredible cathedral-like church in a low-income village with a population of 1,500. At one time it was a major port but was stymied by politics and the changing geography of the East coast waterways. It's left with the plan and accoutrements of a successful medieval town but is now a village with little self-income.

In other cases there are towns like Beverley which rapidly outgrew its town ditches as its centrality to the wool and grain industry coupled with the importance of its connection to the River Hull (effectively a commuter route to the North Sea trade network) gave it a sudden wealth boost in the early 1200s. Later ditches were built to encompass the town but in all it demonstrates the 'usual' pattern - high wealth buildings (mints, warehouses, armouries, high-status residences) were kept within the boundary while the low-wealth buildings spread outside.

Sources:

Mick Aston: Interpreting the Landscape and Local History (1985)

Mick Aston: Monasteries in the Landscape (2000)

Barbara English: The Lords of Holderness (1977)

A.G. Vince and John Schofield: Medieval Towns - The Archaeology of Towns in their European Setting (2003)

So debate has raged for decades as to the precise nature of the ninth/tenth century burghal networks and their precise intention and development, largely because there are always outliers to every theory. For some in-depth reading, I'd recommend Martin Carver's work on Stafford, Burh to Burrough, Christie and Creighton et al's Transforming Townscapes - From Burh to Burrough: the Archaeology of Wallingford AD800 - 1400, Owen-Crocker and Thompson's Towns and Topography, Baker and Brookes' Beyond the Burghal Hidage and Baker, Brookes and Reynolds' Landscapes of Defence. My own thesis which is tangentially on the subject is available here.

The problem with trying to classify burhs is that there's such a broad series of sites that there are always exceptions to any given theory. Indeed, given prior and later use of 'burh' to donate just an enclosure rather than a state-mandated fortress, John Blair argues that we shouldn't even use the term burh at all. At a purely basic level, burhs were indeed 'just' fortifications - well protected garrison sites designed to defend Wessex and Mercia from the Viking threat. Situated roughly 40 miles apart, burhs were designed to curtail the war of manoeuvre and raiding which had characterised Danish warfare in Europe for almost a century. Burhs are typically situated at important transport or infrastructure nexi: major bridges and river crossings, road junctions, prominent landing sites and the like, specifically to control those points and to deny their use to an invader.

For a long time, it had been thought that the burh's role was predominantly that of a refuge. Situated at focal points, often with a large, open quarter, and no more than a day's walk away, it had been presumed that burhs were intended for the local population to flee to in the event of an attack. This theory has been radically overturned in recent years, particularly by the work of Baker, Brookes, Reynolds, Creighton et al. A predominant problem was that 20 miles, whilst feasibly walking distance, is a significant hurdle when burdened with children, the ill, infirm and elderly, and while trying to drive the cattle and carry the valuable personal effects that would be desired loot by any attacker. Based in part on a body of work examining local fortifications including the re-use of hill forts, and the establishment of watchtowers and signalling stations, the burh has been re-conceptualised from mass refuge to something resembling what a modern military might call a Forward Operating Base.

In their new context, burhs function as the linchpin of a wider defensive network predicated on local defensive strongpoints supported by a series of observation and signalling stations. We know from archaeological evidence - particularly in Mercia where the ceramic evidence is most definitive - that there were small garrison forces established across these local strongholds. These can often be identified by weard toponyms - the same Old English from which we get ward, warden etc. - and are often either small towns with palisades or even Roman walls, refurbished hill forts and the like. Leintwardine, Burwardsley and Wardlow are all good Mercian examples. These provided quick refuge to local populations while word was sent via beacon or messenger to the nearest burh or burhs, who could then rapidly dispatch their garrison forces to meet any attackers in open battle, where Anglo-Saxon shieldwall tactics were most effective. Just as the burhs denied the use of roads and rivers to the enemy, English fyrd garrisons could in turn make rapid use of them to outmanouevre an enemy and force them to battle on English terms. This system depended on the constant presence of the fyrd, a garrison force that was essentially part peasant levy, part standing army that worked on a rotational system and was highly effective. The effectiveness of the system overall can be seen particularly in two episodes: the Viking attack on Chippenham in 878 was designed to coincide with another on the coast of Devon. Alerted to the attack, the fyrd of Exeter was able to destroy this second Viking force in detail, and as a result the South Coast county fyrd provided the bulk of Alfred's victorious army at Eddington. In 911, a large force from the Danelaw attacked Mercia, as the royal army of Edward the Elder was in East Anglia. Forewarned, the Mercian fyrds were able to ambush and destroy the invaders at the battle of Tettenhall.

Maintaining such an extensive series of fortifications and the fyrd garrisons to man them was an enormous logistical and economic undertaking, and is arguably central to the rapid unification of England under a centralised power, and it's this which heavily affects the development of burhs into towns. Building new fortresses from scratch could be an incredibly expensive project, so whenever possible, Alfred and his children made use of what was already in place. Luckily, major river crossings, landing sites and road junctions tend to be the sort of sites that already attract settlements. Some, like Exeter, still had intact Roman defences while others, like London or Chester, could be repaired, and so these became prime burghal locations, which then benefitted from the existing population. Major trade ports like Hamwic were already substantial settlements in convenient locations. At Tamworth, the burh-ification entailed the expansion and extension of the pre-existing fortifications of the Mercian capital. In many cases, the establishment of a burh at these sites still involved a significant undertaking, with the construction of major defences and often the institution of a new street plan with an empty quarter established to garrison the rotating fyrd forces.

For sites that were built de novo, a variety of factors influenced burghal survival and development. In many cases, burhs were explicity intended to become proto-urban sites. The safety of a burh, its prominent location and the necessary presence of administrative officers for its continued function meant that a wide variety of civil and social functions were moved inside the burhs. They became the centres of judicial and legal offices, the base of royal officers and centres of the Church. Fortified garrisons offered a badly needed sense of security for vulnerable religious communities and also security for precious relics. Æthelflæd famously mounted a military expedition to Danish-held Bardney to capture the relics of Saint Oswald and translate them back to the newly re-fortified Gloucester, and in 908, the relics of St Werburgh were translated from their home at Hanbury, within visual distance of the Danelaw settlement at Derby, to the new safety of the burh at Chester.

Burhs could become economically important too. The Anglo-Saxon economy was heavily monetised, but minting was also a major propaganda tool. Successive laws from the late ninth century onwards dictated that moneying only take place in burhs, where it could be more heavily supervised and controlled by the presence of royal officers. Æthelstan's laws of the 920s also dictated that any transaction over the value of 20d be carried out in a town and witnessed by relevant authority; in many cases this would have been a burh.