We know quite a bit about this topic, though not (of course) as much as we'd wish. Your intuition is correct: there were not miles-long wagon-trains full of silver threading through the countryside every time a Roman aristocrat bought a new estate for 8 million sesterces.

By the 2nd century BCE, the concept and deployment of credit was well-established in the Roman world (and Hellenistic Greek world, as well). Large-scale transfers of money were always in credit, never in cash or kind. By this time, the Roman elite had made a business out of lending money at interest, and the assets of many of the wealthiest Romans were tied up in massive loans as much as in land. These were "private" loans in the sense that they came from a single individual or his family, not a banking institution, and the vast majority were "consumption" loans--they came about because the other party needed help (as opposed to capital investment). That is, they were non-productive loans as far as modern economic thinking. Their profit was from the interest, and the principle had no potential for growth beyond that. Many of these loans extended wealth from Italia into the provinces. Famously, the Gallic revolt in 21 CE was supposedly caused by masses of overextended borrowers (Tacitus Annales 3.39-41). There seem to have been few regulations on this private loan industry, and lenders (like the famous M. Licinius Crassus) could be unscrupulous in collecting. Cicero suggests several times in his In Cat. speeches that Lucius Sergius Catilina's real reason for plotting a coup and tabula rasa (cancellation of debt) was the impending due dates on his own massive loan obligations.

A lot of scholars argue that the Roman economy, unlike modern economies, was not equipped to deploy capital for growth. The super-elite of Italia had millions and millions of sesterces and no real way to deploy that wealth to generate more wealth. I'm not so sure about that, but I'm not nearly smart enough to understand the intracacies of economy either ancient or modern. There were definitely "organizations" that pooled wealth to achieve financial feats not feasible or wise for individuals. This sort of group was called a societas. The famous one is Cato the Elder's societas, which might have had as many as 50 members, of varying wealth, working towards their common goal to finance maritime commerce in some way (Plut. Cato Maior). I myself am not sure if Cato's group distributed loans to shippers as a group to minimize their risk, or if they were deploying their pooled capital to create new ships.

Loans were made in a variety of different ways. Some were straight up person-to-person, especially when between two elite friends. The vast majority, however, were through intermediaries, professionals, who took a cut of the principle or the interest in exchange for their services. These men were called negotiatores, and they usually intermediated loans along with a slew of other financially-related tasks. These are the ancient brokers. There were also professional financial slaves or freedmen, called faenatores, who handled the finances of their masters/patrons. Both of these types were probably preferred by Roman elites because they knew the business well, and could also "talk business" (ie be a little rude) without risking causing offense. In fact, I think it is very surprising that Cicero "talks business" as much as he does in his letters. Both he and his filthy rich friend Atticus employed negotiatores (Cic. Letters to Atticus 6.2.3).

There were also "bankers" (argentarii). From what I've written above, I hope I made it clear that bankers were not necessarily needed for large-scale transactions. The bulk of money changing hands in the 2nd and 1st centuries BCE did not go through the argentarii. Nevertheless, these guys came to offer all the same services as your typical bank today. They accepted deposits, payed depositors interest, and energetically invested those deposits with third parties. They offered the same sorts of general financial services and know-how as a negotiator, but usually on a smaller scale. We have some evidence that they secured important documents, so at least some of them must have maintained guarded physical buildings. The argentarii also often seem to be involved in the legal world, either as executors of contracts or as facilitators in obtaining other executors. This is a confusing world, because it is usually impossible to tell if a mentioned person is a negotiator, a faenator, or an argentarius. Their activities were very blended, and most of them seemed to wear several hats at the same time. I myself think that there is a social element to these different roles, and a lot of work has been done recently on that topic. I suspect the argentarii were, by the end of the 2nd century, strongly "Italian" in character (that is--from the cities of Italy, the "allies," not Romans).

Sources. For this topic, the best is probably J. Andreau, Banking and Business in the Roman World (Cambridge 1999). To read about the Sulpicii, a very famous group of negotiatores/argentarii, see D. Jones, The Bankers of Puteoli: Finance, Trade and Industry in the Roman World (Stroud 2006). In general, see The Monetary Systems of the Greeks and Romans (Oxford, 2008), especially 174-207.