I'm curious about anywhere in the Empire. And regarding huts vs houses, I'm considering huts to be something tiny that's built with the intent that it'll be torn down and put back up again when it gets leaky rather than something built with the intent of being a permanent structure for generations.
The size and prosperity of Roman villages varied considerably, as did the design and solidity of their houses. The dwellings in some Roman villages, however, were durable enough to last for centuries.
First, a brief note on definitions. For the Romans, "city" was a legal designation. A settlement was a city not necessarily because it was the biggest and richest settlement in its region (though this was usually the case), but because the Roman government had recognized its possession of a certain set of institutions and rights. Villages, by default, were all settlements - some quite large and prosperous - that lacked this designation. For the sake of simplicity, however, let's just take village to mean any relatively small and predominantly agricultural settlement without monumental public buildings.
As you might expect, housing styles differed considerably from region to region, in keeping with the local climate, materials, and building traditions. Once - to mention an extreme example - I visited a set of Roman houses (in Bulla Regia, Tunisia) built underground to escape the pervasive summer heat. As Vitruvius (author of the only extant ancient treatise on architecture) observes:
"It is not in the power of an architect to control the quality of the materials [in a house], for the same species of materials are not found in every place; and it depends on the client whether the building shall be of brick, of rough stone, or of squared stone." (6.8.9)
In some parts of the Empire, most villagers lived in huts built of perishable materials. For centuries after the Roman conquest, most inhabitants of Britain continued to live in circular wooden huts with thatch roofs.
In other regions, however, dwellings could be substantial, multistory stone structures. The best-preserved examples are found in the so-called Dead Cities of northern Syria, a cluster of late Roman villages deserted in the Middle Ages. Check out these impressive houses in Serjilla, one of the largest Dead Cities. They are thought to have been inhabited continuously until the city's abandonment, perhaps for a half-millennium or more.
[At the moment, I'm in the middle of a hiking trip, with very limited internet access. If anyone has any follow-up questions over the next couple hours, I'll be happy to address them. After that, however, I'll be internet-less for a day or so.]
The answer, to a certain extent, is both yes and no. It's very difficult to answer such a broad question, because you're talking about several hundred years and thousands of miles. My personal expertise is in early medieval Europe; others can speak more specifically about other parts of the Empire, such as North Africa or the Byzantine empire.
Only truly nomadic peoples living in what had been the Roman Empire, such as the Berbers of North Africa or the Bedouins of the Middle East, would have had truly portable housing that they took apart and put back up again regularly. You might also see portable housing amongst populations of, for example, sheep or cattle-rearing groups, who had to take their flocks to regional pastures. Most people in western Europe in the late Roman and early medieval periods lived in small, but relatively settled, villages.
Especially when you are talking about the peasantry, lifestyles of the middle and lower classes were very slow to change except in cases where some major event such as war or disease affected population levels or the ability to exist in a certain place. The dissolution of the Western Roman empire was a very slow process, administratively speaking. If you read the writings of Gregory of Tours, who died in 594 AD, you will still find many references to "senatorial families" and the old Roman power structure, which still had significance even in firmly Merovingian France, a century after the last Western emperor was dethroned.
The Roman Empire was massive, especially in an era where fast travel and quick communications were difficult or impossible. There was an enormous amount of regional and cultural variation in what local villagers did and how they lived; however there was a great deal of continuity from the Roman Empire through the early Middle Ages.
In post-Roman England and Wales, many families lived in thatched cob or wattle-and-daub cottages. Roman villas which were still in good condition were sometimes partly re-occupied by one or more families in later eras before being abandoned, still others re-occupied other, older Iron Age structures in places where they existed. In Northern Europe in general, change came very slowly to the middle and lower classes after the Western Roman Empire dissolved, certainly until the chaos of the Viking age started around 800.
For most of the peasantry of post-Roman Europe, though, they would have lived in buildings made of what they had to hand, which they would have repaired as best they could until the buildings were beyond repair. Frequently, they would take apart an older, crumbling building which was no longer needed, and repurpose the stone or materials to build something new, whether a house or a stable or a church. This happened to parts of Hadrian's Wall, for example, but examples can be found all over the former Empire. This repurposed material is called "spolia" and you can find information on it by searching up that term.