Yes, that is definitely possible, and in fact in some places (and at certain times) it will probably be the usual solution (and maybe even the only one). A fine might also be the punishment for major crimes - even for murder/homicide and major theft, you could be punished with a fine.

Fines go back much further than 1100. Hopefully someone more knowledgeable of earlier medieval periods will weigh in here (BRIStoneman?), but this is the basic concept of a wergild. Everyone had a price, and if they were injured or killed, that was the price the guilty party would pay in compensation. It depended on your age, status, gender, etc…a man, a woman, a child, a slave, a nobleman, or a peasant were all “worth” different prices. The wergild developed as a way to avoid the endless cycle of vengeance that would happen if someone was assaulted or killed - they or their family would attack the other side, those people would attack back, and so on, forever and ever. Wergild meant that the assault/murder was solved right away, with no need for further violence.

If you were the guilty party, and you couldn’t pay the wergild, then hopefully your family would pay the fine for you, or if not your family, then some other person. It didn’t matter who paid the wergild, as long as it was paid. If you couldn’t pay, and no one could/was willing to pay for you, then the victim (or their family) could take vengeance after all.

Wergild is more of an early medieval thing, as far as I understand, and it fell out of use as more centralized governments and legal systems developed. But it seems like it was actually just redefined as a fine to be paid to the government, rather than to the victim directly. Everyone still has a price, sort of…for example, I am thinking of one specific law (from crusader Jerusalem, of course!) where the punishment for a knight who assaults another knight is 200 bezants (i.e. Byzantine gold coins), 100 to the king and 100 to the victim. But if the knight assaults someone who isn’t a knight, the fine is only 100 bezants (50 to the king, 50 to the victim). That still seems like it’s a kind of wergild, isn’t it? It’s just not called that, because it’s the fine that the court charges the guilty person, and it’s no longer meant to stop an otherwise endless cycle of vengeance, as it was in earlier times when governments were less centralized and there was no state legal apparatus.

So, again, now what happens if you can’t pay the fine? Karen Jones gives some evidence for late medieval Sandwich in England:

“the problem of defendants who could not afford to pay was resolved by having the fine paid by others. Twelve masters were recorded as paying fines incurred by their servants for assaults; in eighteen cases the sureties paid; and two men paid the fines of male relatives. For the nine women known to have been fined, payment by the husband was recorded in three instances." (Jones, pg 92)

This is for just one place in the 15th century though, and there is no general rule for everywhere during the whole period. And as Jones notes later on, fines were sometimes gendered, in the sense that men were more likely to be fined, and women might have to do something else, since they typically didn’t own any property or money; they could be publicly shamed for example, although men could be shamed in public too.

Like the wergild, if you couldn’t pay the fine, ideally someone would pay for you. In this case as well, it didn’t actually matter who paid it, as long as it was paid. You might be in debt to someone else if they paid the fine for you, but that was another matter.

You mentioned that imprisonment was uncommon, because prisons were expensive to maintain. You’re right, and the idea of simply imprisoning criminals is more of a modern development. But prisons were used sometimes in the Middle Ages and especially in cases where someone couldn’t pay a fine. Trevor Dean notes that people in the royal domain in France could be imprisoned if they were unable to pay fines in the 13th century (which is pretty early for the use of imprisonment). But you wouldn’t just rot in jail forever; the prison term could be symbolic, maybe just a day or a week.

So, in brief: if you couldn’t pay, the best case scenario was that someone would pay for you. If that didn’t happen, maybe you could be imprisoned, or publicly shamed, maybe you might be the victim of revenge, depending on the time or place.

Here are some sources, although there could be an almost endless list of sources for all kinds of different times and areas:

Trevor Dean, Crime in Medieval Europe, 1200-1550 (Routledge, 2001)

Barbara A. Hanawalt, Of Good and Ill Repute (Oxford University Press, 1998)

Barbara A. Hanawalt and David Wallace, Medieval Crime and Social Control (University of Minnesota Press, 1999)

Karen Jones, Gender and Petty Crime in Late Medieval England (Boydell, 2006)