In Acts 19:18-20, Luke mentions that a group of "magicians" in Ephesus burn 50,000 denarii-worth of "magic books" when they convert. What kind of magic would be practiced in first century Ephesus?

That's a potentially very broad question, but I can try to answer it by discussing what kind of "magic books" would have been burnt and their contents. For a little wider background, Owen Davies in Grimoires: A History of Magic Books 19 writes:

As related in Acts 19, many of the Jews and Greeks of Ephesus, a city now in Turkey, famous for its Temple of the goddess Artemis, 'used curious arts [and] brought their books together, and burned them before all men: and they counted the price of them, and found it fifty thousand pieces of silver.' Ephesus had a reputation as a centre of magic in the Hellenistic and Roman world, and the term Ephesia grammata (Ephesian letters) was given to mysterious non-Greek and non-Latin words, supposedly engraved on the statue of Artemis, which were thought to have 'evil-averting' magical properties and were used in magic inscriptions and curse tablets. A silver piece or drachma was equivalent of a day's wages at the time so, whether the New Testament story is factual, the intention was to show the value placed on the magic books in circulation in the city. Some Bible scholars have suggested that those who handed over their magic books were newly converted Christians destroying the last vestige of their old religion, though the dominant interpretation is that it was Jews and pagans who handed them over as an act of conversion to the new faith. Either way the sheer number of books involved, if the details are to be believed, suggest that grimoires were not only in the possession of the professional magicians and healers but were kept in many homes.

As for the contents, you're probably looking at a miscellaneous collection of charms, incantations, recipes, and the like taken from various occult traditions; the Greek Magical Papyri probably give a good indication of some of the contents of the hypothetical bonfire, and includes a syncretic mixture of materials featuring elements of Greek, Egyptian, Babylonian, Jewish, and Christian mythos.

It's really difficult to say what else might have qualified besides manuscript collections of spells and charms. They might have included books of prophecy; medical texts that contain charms, prayers, or incantations; Pliny's Natural History; or various non-canonical* Jewish, Gnostic, and early Christian works like the Book of Enoch in the Dead Sea Scrolls (Davies 8) or the Testament of Solomon.

Manuscript collections of spells and charms show a vast variation of recipes and incantations designed for pretty much every worldly need. Many of them were medicinal, divinatory, or concerned with winning love, preventing pregnancy, finding treasure, securing victory in a legal suit, curse your enemies, craft amulets or talismans of protection, etc. A number of different degrees of divination were practiced in the Roman empire, from astrology to dream interpretation. Goetic magic would involving the summoning, commanding, or dealing with spiritual entities like demons, angels, devas, etc., and exorcism would be its natural "anti-magic" counterpart.

It's probably worth mentioning that the more harmful forms of magic were outright illegal in the Roman empire; there's a really decent article on it by Pharr called The Interdiction of Magic in Roman Law, and there's a number of other sources that can expand on that point if you're interested, like the early chapters of Edward Peters' The Magician, The Witch, and the Law (which mostly deals with a medieval context, but starts with a look at antiquity.) These also give an idea of some of the types of magic practiced during the period.

Caveat: In the early Christian church, there was no set canon (or rather, various different standards of what was canon) with regards to the gospels as New Testament literature developed; among Rabbinic Jews the Tanakh canon was still being defined into the early Christian era; various manuscripts from both sources have included occult elements and references. So when I say non-canon I mean largely by today's standards, not necessarily the contemporary standards of 100-400 AD.