Your assumption that there were no "true banks" in the ancient world is erroneous. By the 1st century BCE, Rome (and the larger Hellenistic world to which the city belonged) enjoyed a sophisticated array of fiscal systems very similar to those employed by modern banks.
When you deposit your paycheck in a bank today, you aren't receiving a pile of coins or notes and then carrying them in a bucket to a bank, which will store your pile of coins/notes in its vaults until you need it. It is a cashless transaction, backed up by a series of laws ensuring contractual fidelity. It was much the same in Rome (for these purposes, let's stick to the 1st century BCE). The major real difference was scale. There was no machinery in place to buy a loaf of bread with a cashless transaction (like somebody buying a soda with a credit card today). Those sorts of transactions were on large scale (buying 3000 acres of land; or financing a new apartment block; etc). The small-scale economy was cash (coin): that's how soldiers were paid, how day-laborers were paid, and it was coins that circulated in the shops and brothels ("street level"). These were sometimes gold or silver (and therefore backed up by their own intrinsic weight value), but also bronze coins which were representative of value--what we call "fiduciary" coin. The latter category is only possible in an environment which guarantees fiduciary value over time and space.
The institutions of the cashless transactions were not Roman innovation, however. Most of the structures originate in the Hellenistic (Greek) east. There were groups ("banks"; Latin mensa, Greek trapeza) for offering and ensuring loans, exchanging currency types, making and safe-keeping deposits, managing payment plans, etc. These might not always been within the same group, but all of them were subject to a standardized system of underlying contractual law. There were witnesses for contracts (syngrapha).
The richest guys, like Crassus, did not sit on huge vaults of coins like Scrooge McDuck. In fact, it would have been impossible for Crassus to liquidate his wealth. I suspect there wasn't enough coins in the whole Med. to accomplish that. His wealth was tied up in assets, the list of which was probably enormous: property first and foremost, and then the buildings, the slaves, the produce and manufacture from it; art; speculative loans and interest incomes; mining contracts; shipping; etc etc etc.
There was wealth in the world, of course--physical piles of gold and silver, and I'm sure Crassus had some vaults here and there. In Pompeii, excavators have found several objects which might have been "safes" or strongboxes, though that interpretation is not always crystal clear. The famous House of the Vettii had a couple in its atrium, and don't let the size fool you: that box, half full of silver coins, represents quite a bit of wealth.
The Roman state herself was an entity which frequently dealt with physical wealth (Pompey carried back thousands of pounds of gold and silver from his war with Mithridates in the East). The home of the State's vault was under the temple of Saturn in the forum. Caesar famously went straight to it when he brought his legions into Italia in 49 (soldiers want coins in their pockets, not little pieces of paper that promise coins later--that couldn't buy you wine and whores).
This topic is covered well by von Reden in the chapter "Money and Finance", Cambridge Companion to the Roman Economy, ed. Walter "The Man" Scheidel, CUP 2012.
In addition to OP's question:
I have read that Ancient Rome had the concept of Corporations or limited liability Companies. Is this true? If so, how did it differ from Corporations and Joint Stock Companies that came up during the Industrial Revolution?
Followup- why didn't anyone thought about creating such banking system? RE had every factor that supported such structure- good communications, immense fortunes, huge territory. Were there any attempts of creating something like that?