/u/xenophontheathenian's linked answer puts forth the nuts and bolts fine; I don't have anything to contribute to that. But I thought an illustration might help make his explanation more manageable.

Cicero was governor of Cilicia (in Anatolia) in 51-50 BCE, and wrote a series of letters to his friend Atticus, which we happen to have (this is the first letter; you can click through to read the rest). In these letters he describes a difficult tax situation that shows us how publicani operated and integrated with the Roman government to extract taxes.

I've linked the letter, and it's worth hopping over to read it yourself, but the letter is long, and Cicero covers other topics with his friend as well. I'm going to talk through Cicero's story here.

The guy who was governor of Cilicia before Cicero was Appius Claudius Pulcher. He was a brother of the tribune and sometime Caesarian Clodius, and head of the family. He had been consul in 54. Cicero describes the province as a patient, and the governor as a doctor. Appius Claudius treated the patient with a course of bleeding, he says; he stripped everything and handed over to Cicero a patient in a state of exhaustion. Here's Cicero's description of the two administrations:

For what could be a more striking difference—under his rule a province drained by charges for maintenance and by losses, under mine, not a penny exacted either from private persons or public bodies? Why speak of his praefecti, staff, and legates? Or even of acts of plunder, licentiousness, and insult?

The tax assessed under Appius was indeed beyond the means of the people of the province to pay, and in order to do so they borrowed money from their Roman friends. Cicero talks about Pompey, and in particular Brutus, as lenders. Brutus had loaned the Salaminians (on Cyprus) a chunk of money at a generous rate of 48% interest. Brutus had got two Decrees of the Senate to enforce the collection of this debt. He had been collecting through his agent Scaptius, whom Appius had appointed to the position of prefect in his staff. This position equipped Scaptius with soldiers, which he used to besiege the recalcitrant senators of the city inside their own senate chamber, during the course of which five senators died of starvation.

When Cicero took over, he ordered the troops off the island and capped interest at 12%. The decrees Brutus got Cicero lawyered his way out of. Scaptius lost his position, and Brutus wrote Cicero asking him to appoint Scaptius to a prefecture on his own staff. Cicero refused to appoint Scaptius as he refused to use tax collectors on his staff, and also enforced his 12% cap on interest. He arranged with the Salaminians to pay back the loan at 12% in a lump sum, but Scaptius refused to accept this amount.

Brutus got Atticus to lean on Cicero in an effort to get Scaptius troops again. Cicero rather uncharacteristically showed some spine and refused again, making quite clear that 48% was a shocking level of interest.

In the end Brutus got his loan and 12%, about half what he expected.

The episode demonstrates how Romans exploited the provinces: rich senators loaned provincials money to pay exorbitant taxes at interest. To collect the debt they used their connections with the senators governing provinces to use Roman soldiers as muscle. If a governor was uncooperative, they would use the senate to order the governor to comply. In the case of the Salaminians, they were lucky Cicero was sympathetic, allowing only 12% interest, and a good lawyer.

Not to discourage further discussions, /u/XenophonTheAthenian provides an answer here.