Roman slaves only held personal property (peculium) by their master's sufferance. The nature and value of this property varied with the nature of the slave's duties; herdsmen, for example, might "own" a few of the cows they tended. But the only slaves who could become anything like wealthy were the relatively privileged minority who worked in cities, and in close association with their masters.

Although they had certain rights over their property, it legally belonged, like themselves, to the master. A master who wanted to strip a slave of his savings could do so. But custom weighed against this, as - in many cases - did economic logic. From a master's point of view, a slave was nothing more and nothing less than an investment. To get the most out of that investment, he needed to motivate the slave to work as hard and as well as possible, ideally without supervision. Allowing a slave to earn money and buy his own freedom was a powerful motivating tool.

Masters freed their slaves on different terms - sometimes in their wills, sometimes when a slave reached a certain age, and sometimes when a slave purchased his own freedom (normally at a price more or less equivalent to his own market value). In the city of Rome, slaves seem to have usually earned this money by operating - in an independent or semi-independent capacity - some business enterprise on their master's behalf (they might serve as doctors, or sell high-value craft goods from "private" storefronts). Masters allowed them to keep a portion of the profits, and to apply this portion toward the purchase of their freedom, for the simple and sufficient reason that slaves working toward freedom tended to work much harder than slaves laboring without hope. In these instances, masters refrained from plundering their slaves for the same reason: it was bad for business.

Roman slaves, of course, seldom became rich (though they might be put in charge of valuable property - including other slaves - in the interests of the master's enterprises). It was freedmen - former slaves - who accumulated really tempting fortunes. One thinks of Trimalchio, the freedman in Petronius' Satyricon who is so rich that he has hundreds of his own slaves, and only knows a tenth of them by sight. Or of the emperor Claudius' freedmen, who became some of the wealthiest men in Roman history. Why didn't masters swoop in and claim the riches of their former slaves?

Manumission did not sever the relationship between a Roman master and his slave. It simply changed the terms of that relationship. A freed slave took his master’s praenomen and nomen as his own, and remained part of the family (in the Roman sense of familia - i.e. the master’s household). He became, in effect, a sort of son. He continued to have obligations to his former master, which sometimes included unpaid labor and always entailed social deference. And the erstwhile master, for his part, became the freedman’s patron. These ties lasted as long as the freedman lived, and even beyond (freedmen were sometimes buried in their masters’ family tombs). The relationship obviously favored the former master; but it also placed him under real social obligations.

Masters did have a legal claim on their freedmen’s wealth, in the sense that they (or their heirs) usually had to be included in their former slaves’ wills. Roman slaves could not make wills; but freedmen were citizens, and had the right - within certain limits - to pass wealth on to their heirs. The property of a freedman who died intestate automatically devolved to his master (or master’s heirs). Freedmen who made wills still had to leave half their property to their former masters if they had no natural heirs; and even if they had children, only three or more entitled them to leave their masters out entirely.

But although they maintained rights over their former slaves’ estates in death, and might demand valuable services from them in life, former masters were not motivated to openly pillage the wealth of freedmen who made it big (though freedmen were expected to help masters who had fallen on hard times). The pseudo-familial terms on which their relationship was conceived militated against blatant seizure of wealth. To most masters, in any case, a wealthy freedman was valuable in his own right - an advertisement and extension of the master himself. Social capital, in short, usually mattered more.

[Edits: added some context, cleaned up typos. My thanks to the commentators on this response, who raised many thought-provoking points.]

If I'm understanding you correctly you're asking about slaves buying their freedom and whether their masters could confiscate their wealth while they were slaves, not after they had already become freedmen. In that case /u/toldinstone hasn't actually answered the question.

Slaves couldn't attain great material wealth, and strictly speaking they couldn't use their own wealth (which they didn't have) to buy their freedom. Slaves could not own property. Instead, like sons in potestate they could be given a sort of "stewardship," called a peculium, that by a series of legal fictions was treated as the slaves property even though it contributed not to the slave's wealth but the master's. This leads to a number of complex and ingenious legal solutions regarding liability, but the bottom line is that when slaves "bought" their freedom it was because they were allowed to by their masters. In practice, and for most legal purposes, the slave owned the property, which might be money or might more commonly be administration over a business or shop. But legally speaking he was increasing the master's assets until he reached a predetermined or generally appropriate level, at which point he formally returned his peculium back to his master and was expected to be freed. There was no legal requirement that a master go through with it, or that a master allow his slaves to hold peculia. In reality even the most humane masters seem only to have allowed a small number of their slaves to have peculia, and only a small number would be in a position to increase the peculia. Further, a peculium could be revoked at any moment, or a master could arbitrarily decide that the increase to his assets was not sufficient. This does not seem to have happened much, simply because a master willing to grant his slaves these concessions in the first place and whose slaves felt that they could even try to become manumitted by buying their freedom was likely to be the sort of person to respect such an arrangement, since if he just wanted to make a quick buck off his slaves there were far more exploitative and effective means of doing so. Still, strictly speaking it could happen, and no doubt there were arbitrary and tyrannical slaveholders who led their slaves on for their own gain, though at least regarding the peculium specifically there aren't many traces of this in the texts.

Also we’re slaves property of their masters? Or were they property of Rome itself? Because when slaves bought their freedom, it wasn’t like a thing where they paid their masters and their master freed them. They had to go to an office that was ran by Rome and they paid there to attain their freedom paperwork. Even if the master wanted to free one of his slaves he would have to go to the office with the slave and buy his freedom for him.

Idk, it’s just confusing to me. It seems like the slaves were property of Rome itself rather than the master.