How extensive ... is a really hard question for scholars studying eras like the middle ages when statistics weren't important. What we can say, however, is that slavery was endemic in early medieval Europe, and that the slave trade was especially prevalent around the Irish Sea during the centuries preceding the Norman Conquest (1169).

The critical study for understanding this period is Paul Holm's 1986 article, "The Slave Trade of Dublin, Ninth to Twelfth Centuries." Holm cataloged 23 annal entries referring to slaving between the years 821 and 1032, which includes the period of Brian Boru (c.941-1014). Holm argues that Irish and Norse-Irish (aka "Viking") leaders regularly engaged in slave raiding. They saw it as a good way of both asserting your authority (Look what I can do!) and subverting the authority of your enemies (Don't be loyal to that guy—he sucks!).

There's a few important things that come out of his article, and a few caveats that have been made during the 30 years since its publication. First, Holm drew the conclusion that slaving was not just a "Viking" thing; it was more just a function of local Irish politics. This has an important corollary: religion didn't really matter for slave raiders—power did. The "Vikings"—scholars now prefer the more general term "Norse"—were actually welcomed into Ireland's power struggles, since Norse slave traders would purchase the slaves produced in internal slave raiding, whether by Irish or Norse-Irish lords.

One notable example of this is the Irish Saint Findan, whose sister was captured around the year 800, so he went to ransom her and got kidnapped for his troubles. Regardless of whether he was captured by Irish or Norse raiders, he was eventually sold to Norse slave traders. Findan was fortunate to escape somewhere in the Orkneys, so we know he was being shuttled off into the North Sea, and probably beyond. Scandinavia was developing strong links to Constantinople and Baghdad in this period, so Findan might've escaped a very unfortunate trip south.

Moving forward in time, Holm connected Irish slave raiding in particular with the settlement of Iceland, ca. 874. There's some problems with this. DNA evidence now shows that modern Icelanders have a good share of Irish blood, especially in the maternal bits of their genetics, but we shouldn't think of Norwegian men making a pitstop for brides on their journey into the North Atlantic.

Instead, many Norse-Irish probably emigrated to Iceland, and many of these were probably children or even grandchildren of Norse who had settled in Ireland during the 840s. So the women that the Dublin vikings first brought into their fort? Probably slaves. But the women their sons and grandsons took with them to help set up farms in Iceland? Probably free, and of mixed descent.

Fast forward closer to the time of Brian Boru, and we've got a rare indication that the slave trade continued past the period of Holm's study, at least into the 1060s. Bishop Wulfstan of Worcester took a trip down to Bristol, and he was shocked at the booming slave trade of the Irish Sea. In this case, it seems like the flow of slaves was moving toward Ireland (as it did for Saint Patrick around the year 400). This actually fits Holm's thesis well, that periods of early medieval conflict—in this case, the Norman Conquest of England (1066)—produced a bumper crop of slaves for the market.

There is by the sea a town called Bristol, from which you can cross directly to Ireland, and for that fact this area fulfills the needs of the barbarians. The town's residents, along with others from England, often sail for Ireland. The bishop Wulfstan took from them a long-held custom, for they had become callous to their souls, such that neither the love of God nor of King William had yet been able to abolish it. Indeed, in the hope of great profits they bought up from all of England people whom they sold off to Ireland; and it was their custom first to bed the girls for sport, and then they put the girls out for sale pregnant! You would have cried to see the wretched ranks of young men and women bound together with ropes: these persons of fair appearance and untouched age were at the mercy of the barbarians; daily brought to market, daily offered for sale. O detestable crime! What wretched disgrace! That those bestial and unfeeling men would add to their possessions by selling even their own blood into slavery. (William of Malmesbury, Vita S. Wulfstani, 2.20)

Only later did it become considered un-Christian to enslave women and children taken in war. John Gillingham has written some excellent articles on this. I'd recommend "Women, Children and the Profits of War" (2012) or "Christian Warriors and the Enslavement of Fellow Christians" (2011). David Wyatt also has an excellent entry on "Slavery" in The Oxford Encyclopedia of Medieval Warfare and Military Technology (2010) and has a longer monograph Slaves and Warriors in Medieval Britain and Ireland, 800–1200 (2009), which treats the sources more exhaustively and works as a good complement to Holm's short article.