Some didn't even try to reconcile it. In Candace Wheeler's memoirs, she remembered that her family (who were abolitionists) boycotted cotton in Delhi, NY.

It seemed to me in those years quite natural that we should not wear cotton clothing, because cotton was a product of slave labor, and that the painfully produced linen from flax grown on the farm and spun and woven in our own house should be its substitute.

(Yesterdays in a Busy Life)

The family also used maple sugar instead of cane sugar.

At the time, cotton was rapidly taking over the position that linen had held - the cheapest material, used for linings and underclothes - but linen was still very much around, either as Irish imports (which could be fine or coarse) or through home production. Although the idea that all rural women routinely spun and wove all their families' linen is a nostalgic, mythful one, because rural women were supposed to at least know these skills up until fairly recently before the antebellum years, it's not that out of the question that the knowledge and tools would still be around.

For the wealthier abolitionists, while I can't speak to whether they definitely did or not, it wouldn't be very difficult to avoid cotton. Fine imported linen underclothes would be a status symbol for men and women, and apart from their shirts men generally didn't wear cotton anyway. For women, cotton morning dresses could be easily made of a light wool instead.