At a major trade hub like Rome or Alexandria? pretty much anything you wanted from the Old World. Silk came to the Romans from China, incense from Arabia, spices, ivory, exotic animals, and gems from India. Every year Greek ships left Red Sea ports and followed the monsoon wind (known in west as the Hippalus wind after the first Greek captain to discover it). They then traded gold and silver for exotic goods and returned to ports in Eastern Africa (it was easier to transport the goods by caravan to the Nile and then float them downstream than it was to fight the prevailing north wind on the Red Sea with a full cargo hold).

Domestic goods within the empire also can have a pretty good reach, particularly when the army is the one doing the purchasing. Olive oil, wine, and other Mediterranean goods were regularly transported to the empire's borders at the army's behest. This seems to have created a broader trade network: merchants conveying goods to the army would also be willing to sell at towns along the way to the border, and so goods percolated further into the empire's hinterland than they might have otherwise.

Good ancient sources for this trade are Pliny's Natural History (references to exotic goods are sprinkled throughout, but 6.101 is a classic passage on how Pliny thought the Indian Ocean trade was ruining the Roman economy), and above all the Periplus of the Erythraean Sea, a travelogue of the various East African, Indian, and Arabian ports Greco-Roman merchants visited.

As far as modern sources go, there's a 2011 article in the Journal of World History (vol. 22) called "Provincializing Rome: the Indian Ocean Trade Network and Roman Imperialism" that's pretty good. R. McLaughlin has a new (2014) book called The Roman Empire and the Indian Ocean, but I have not read it and have no idea if it's any good. Of more general interest would be the articles in The Cambridge Economic History of the Greco-Roman World (2007).

Silk has to be one of the top contenders. It not only came from more than 8,000 miles away through some of the most inhospitable and dangerous regions on the planet, but the method and means of its production was a state secret of every Chinese dynasty for millennia. They took that very seriously... Through the Han, Tang, and even through the Yuan it remained a jealously guarded secret and being caught trying to smuggle live silk worms out of China was a virtually automatic death penalty for all involved. Japanese smugglers managed to finally get some living silk worms back to the home islands around 300 CE, as well as four girls who were forced to teach them how to properly care for them through their productive life cycle. But they wouldn't be successfully smuggled to the West until at least the 6th century when, according to Procopius, Roman Emperor Justinian I at last procured several live eggs, thanks to the efforts of a pair of Nestorian monks/smugglers. Those thieving, tonsured bastards.

Still the idea of silk being a solely Chinese product is actually where its very name derives. The old Greek name for China was Seres (Σῆρες), which became the name of the lustrous fabric. That in turn became known as sericum in Latin, to sioloc in Old English, and of course now silk.

Even though they ultimately lost their their monopoly on its production, and then the collapse of both the Han Dynasty in the 3rd C. CE and the Western Roman in the 5th, nevertheless the Tang Dynasty of the 7th-10th Centuries managed to re-establish its trade routes with the West, as well as re-estabish China itself as the largest single supplier of the material, and the subsequent Song Dynasty (10th-13th C.) scaled production up even further to an industrial scale dwarfing like likes of the Arabic or Byzantine competitors.

The Roman demand for silk was so high, in fact, and therefore produced such a tremendous outflow of Roman wealth eastward, that the Roman Senate famously tries to ban its purchase and use.. supposedly on moral grounds. From Seneca the Younger's Declamations, vol. 1, ca. 1st c. BCE:

I can see clothes of silk, if materials that do not hide the body, nor even one's decency, can be called clothes... Wretched flocks of maids labour so that the adulteress may be visible through her thin dress, so that her husband has no more acquaintance than any outsider or foreigner with his wife's body.

In spite of - and let's be real here, in many cases because of - it often being seen as both a tremendously expensive and exotic luxury, and its sheer, revealing nature (oh, baby!) on the human form inspiring impure thoughts, it remained a hugely popular commodity even in the face of official attempts to ban it.

By popular request, MOAR SOURCES! ;) :

Bielenstein, Hans. “Wang Mang, the restoration of the Han dynasty, and Later Han” in The Cambridge History of China vol.1.

Dash, Mike. "Wang Mang: China's First Socialist" in The Smithsonian.

Liu, Xinru. The Silk Road in World History

Loewe, Michael. "Wang Mang 王莽 (2)", in A Biographical Dictionary of the Qin, Former Han and Xin Periods

Grousset, René. The Empire of the Steppes (tr. Naomi Walford)

Nishijima, Sadao. "The Economic and Social History of Former Han", in The Cambridge History of China, vol. 1

Whitfield, Susan. The Silk Road: Trade, Travel, War and Faith

Yu, Ying-shih. "Han Foreign Relations: Contacts with the Mediterranean world" in The Cambridge History of China, vol. 1

Follow up question: could I get a banana?

Was the animal trade strongly tied to Rome across the globe, did the fall of Rome stop third party trappers (non Roman trappers, maybe African/Indian trappers) from existing? Were animals transported differently to regular goods?

Which type of fruit was common in the roman empire? They had tropical fruits coming from Asia or Africa?

This is a little later than what you are asking about, and I don't know if these sources are legit enough for this subreddit, but we have instances of Roman glass in Japan, and Japanese glass in Rome, from the 4th and 5th centuries CE.