TL;DR: It probably tasted like dust or whatever you mixed it with.

They were mostly ground up or used in a tincture, if I remember correctly. The tincture could be mixed with food or drink, or taken under the tongue. A 1927 article by Warren R. Dawson says:

"Avicenna (980-1037) describes mumia as useful for a variety of purposes [...] As a drug, however, he never prescribes it alone, but always mixed with some herb, or in some convenient vehicle, such as wine, milk, butter, or oil."

Now that was written almost 100 years ago about the practices of a physician who lived around a thousand years ago. Your question sounds like it's focusing more on Europe in the later centuries. I can't imagine methods would have changed that much. By the time capsules became popular, taking mummia was falling by the wayside.

You might be interested in this article, which talks about different ways body parts were taken internally as medicine. Also, I highly recommend " Mummies, Cannibals and Vampires: The History of Corpse Medicine from the Renaissance to the Victorians," by Richard Suggs. It's pretty gross stuff, until you realize we do similar things today (like stem cell research and tissue transplants), albeit for different reasons.

Sources:

Dawson, Warren R. “Mummy as a Drug.” Proceedings of the Royal Society of Medicine 21, no. 1 (1927): 34–39. https://doi.org/10.1177/003591572702100102.

Dolan, Marie. “The Gruesome History of Eating Corpses as Medicine,” May 6, 2012. https://www.smithsonianmag.com/history/the-gruesome-history-of-eating-corpses-as-medicine-82360284/.

Sugg, Richard. Mummies, Cannibals and Vampires: the History of Corpse Medicine from the Renaissance to the Victorians. London: Routledge, 2016.