Rome absolutely had bankers (argentarii), who follow our concept of modern banking in the broad sense. At least at first these bankers were often located in temples (where coins were minted) but not exclusively. They'd usually be in a fairly informal setting though. They'd have a place to store your money (often at a temple) but not an office building like we'd associate with a bank. They might have a table in a forum. (I say a forum rather than the forum because I don't want to suggest banks only existed in the city of Rome, where people associate the forum with.)

Argentarii took deposits and sometimes paid out interest on those deposits. Interest rates on savings could be highly competitive to attract customers. Bankers would issue drafts for a sum of money a client had on deposit with them (a check). And, perhaps most importantly, they would make loans and charge interest. Although it's important to say this didn't originate with Rome. The Greeks did it before Rome. And I don't know where the very first origin of it was.

Interest rates on loans were enough of a presence in regular life that they are casually commented about in Roman source. See:

Cicero (Att., 4, 15, 7) commented that “interest [rates] went up on the Ides of July from 1/3 to 1/2 percent [per month].” There was “a 60 per cent drop in interest-rates after Augustus brought back treasure from Egypt” (Duncan-Jones 1982, 21).” Source: The Roman Market Economy, Peter Temin Ch 8. P. 170.

These financial structures could be quite sophisticated. For instance, loans were assignable. So, if I loaned you 10 denarii and I died you would owe whoever inherited my estate that 10 denarii. But that also means I might be able to sell your loan and you'd owe the person I sold your loan to 10 denarii. The Roman Market Economy p 171.

It was also common for those setting up new ventures to use borrowed money. So, if you wanted to setup a vineyard you could get a loan with interest on it. Similarly, it was useful for merchants who needed funds to get whatever they sold before recouping a profit (hopefully). So, there were examples of using loans for investment purposes similar but not exactly the same to what you are talking about.

There were even calls to set up a government run and controlled bank although those were not followed up on.

For much of the time period the senatorial class was supposed to stay out of commerce. And some of the wealthy weren't barred from it but would prefer not to be identified with it (bankers are rarely popular). So, it would not be uncommon to provide funds for loans on interest but use an intermediary. Some men would use a freed slave (or an actual slave) for this purpose. So that freemen did begin to be associated with the industry. There were also successful and wealthy freemen in the industry of their own accord.

Ancient Greece had banks before Rome. Some spots were famous for them such as Delos. And there are examples of people taking out loans from the Temple of Apollo by using their house as security (i.e. a mortgage). But I know less about their general setup.

Sources: Rome's Economic Revolution, Phillip Kay and The Roman Market Economy, Peter Temin