I'm not an expert on ancient Rome, so my answer is focused mostly on ancient Greece. In general, though, in ancient times, wealth was based primarily on land ownership. You were rich because you owned extensive stretches of land. This land didn't lie fallow, but was cultivated, either by people in the owner's employ (including slaves) or by farmers who worked the land in exchange for tribute/tax; herds of livestock were also kept (in the Homeric epics, it's common to express the value of something in oxen). It's important to remember that ancient societies were all agricultural societies, and therefore quite different from modern industrial societies.

How land was distributed depends a little on the society in question. In ancient Egypt, for example, land was divided between the monarch(s), the nobility, the temples, and smallholders. Many farmers were peasants who worked the land for someone else in exchange of being allowed to live there and keep some of the produce for themselves. There's a fairly good introduction on the economy in Pharaonic chapter by Christopher Eyre (pp. 291ff) in A Companion to Ancient Egypt (2010) edited by Alan B. Lloyd. In ancient Greece, a lot of land was in hands of the elite, but some smaller parcels were operated by independent smallholders.

Wealth could also take the form of buildings and (portable) valuables stored inside of them. As a more or less generic example for the ancient world, let's take a look at Odysseus' palace in Homer's Odyssey. Whether or not your regard the poem as reflecting any particular period of history, it wasn't considered alien by its audience; it's remarkably consistent (for more, consult Hans van Wees's Status Warriors. Violence and Society in Homer and History, published in 1992).

Odysseus, as the king of Ithaca, has vast estates, including large herds (as do all of the kings and princes mentioned in the Homeric epics). Some of the land is apparently managed/worked by slaves, like the swineherd Eumaeus. His palace, located in town, has multiple rooms. Slaves and servants take care of the house, assisting the king's wife, Penelope.

Odysseus’ house features a store-room filled with all sorts of treasures (keimêlia) and weapons (Od. 2.337–347), and there's no reason to believe that Odysseus is an outlier in this. Weapons, armour, and other valuable (metal) objects are also on display in the king's hall, where the suitors who want to marry Penelope spend most of their time (Od. 19.1–34, 22.23–25). The later Greek poet Alcaeus, who's floruit can be dated to ca. 600 BC, in one of his fragments makes mention of "a high hall" that is "agleam with bronze", i.e. all sorts of expensive weapons (fr. 140 Voigt).

Later, when Odysseus has revealed himself and is attacking the suitors, his son Telemachus runs off to fetch equipment from an inner room (Od. 22.101–115), a kind of armoury located somewhere in the palace, and perhaps separate from the room with the keimêlia. Again, weapons and armour would have been expensive pieces of equipment. As W.R. Connor puts it in his article "Early Greek land warfare as symbolic expression", published in Past and Present 119 (1988), p.16:

These [spoils] may be substantial, given the vast wealth that moves with such an army. The taking of booty was perhaps the largest movement of capital in Greek civic life.

So, in ancient times, wealth took the form of land (and livestock!), as well as portable goods ("treasures", weapons and armour) that were stored inside buildings. There were thieves in ancient times, of course; among the Greeks, the god Hermes was the protector of thieves! Burglary was a feature: the poet Hesiod, a contemporary of Homer, complains about burglars, and it was quite easy to make a hole in the mudbrick from which most walls were made (Works and Days).

But most houses in ancient times would not have been left unoccupied: women were generally more frequently home than men, and there would have been slaves and servants at home, too. Classical Greeks and Romans who lived in larger houses in the city would often rent out rooms that opened up onto the street to be used as shops, meaning there were always people around (cf. the ruins at Pompeii and Herculaneum in Italy).

Before there were banks, temples often performed similar functions, storing wealth and even extending loans. By the Classical period in Greece, there were actual bankers (trapezitai), who worked as money changers (since every city had its own coinage), and they mostly operated at temples and markets. They also held deposits and extended loans. For more, see e.g. W.E. Thompson, “Insurance and banking”, in: M. Grant and R. Kitzinger, eds., Civilization of the Ancient Mediterranean. Greece and Rome, vol. 2 (1988), pp. 829–836. (As an aside, free citizens usually avoided becoming involved in banking, leaving this business to metics in particular, i.e. resident foreigners.)

Archaeologists also discover hoards every so often, which are valuable items that have been buried in the ground. The original owner clearly never had the opportunity to retrieve what they buried. We won't know why they buried this stuff; perhaps they wanted to avoid burying taxes. Wikipedia actually has a useful overview that deals with hoards, so you can get some idea of what they contained.