The reason is that the cities of the empire depended on the state to shuffle resources into them, both in the form of government grain ships / caravans, and the merchants that use the roads or piggyback on the ships.

With the collapse of the empire, the ships stopped sailing and the roads begin to fall into disrepair. This means the resources needed to sustain the city's population were no longer flowing into the cities. The population that could no longer be fed therefore naturally dispersed.

EDIT: Home now.

Besides the above, there's also a decrease in temperature in the late Roman Empire that coincided with a decrease in size of agricultural estates, the estates which fed Rome's cities. Change in temperature also resulted in crop-failures and famines.

The plague also cut into the population. A decrease in population means less land could be cultivated.

Another thing which happened (that might have been a result of the fall of the empire) was the fracturing of the empire's areas into more isolated economic zones. The farm estates of antiquity that were specialized for exporting goods became more regional (and also shrank in size). started planting other things like vine. This was probably both caused by and further fed into the regionalization of Europe as large bulk goods like grain no longer flowed through trade but became stayed in the local area, decreasing the ability to feed large cities. Justinian's reconquest of North Africa and Europe temporarily reconnected the trade networks, but it was soon severed again by the Arab conquests.

One thing pointed out is that the cities of the Roman Empire was built as administrative center by and for the state, to collect taxes, distribute supplies to both the population and, most importantly, the army. At military cites were found:

einkorn, polish wheat, durum, spelt, wheat, parley, rye, oats, millet, and rice (this last, a clear import!). Local villae on the other hand, produced only two or three varieties of cereal. Rome was able to use for its service a system for the distribution of foodstuffs at world-state level.

So with the collapse of the empire went both the ability and the need for this trade network.

The population slowly and steadily recovered and increased from the 7th century onwards with the coming of the Medieval Warm Period and the slow diffusion of farming techniques. More intensive cultivation and the (re)opening of lands from the 8th century onwards also increased output necessary for cities. Trade also picked up again. Cities (re)appeared as centers of administration but also centers of production and trade (like of wine of cloth), leading to the cities of the 10th to 12th century.

Source is The economy by Jean-Pierre Devroey produced in The Early Middle Ages - Europe 400-1000 edited by Rosamond McKitterick.