Great question! I will touch first on the Russian side of things, though the quick answer is that it is very likely he would have eaten a banana. Certainly by the turn of the century had ready access to bananas in Chicago--the phrase 'banana republic' was coined by O. Henry as early as 1904 in response to American commercial and military interests in Latin America. As an aside, one of the chief architects of American intervention in Honduras, Samuel Zemurray ("Sam the Banana Man") was a Jewish Bessarabian immigrant-turned-fruit magnate and, eventually, president of the United Fruit Company.

But starting in Russia, bananas would be an uncommon sight in the mid-nineteenth century, especially outside of urban centers and the Black Sea region, but not an impossible one. Bananas were known in Russia quite early; a half-century before, the conservative philosopher Mikhail Shcherbatov polemicized upon the presence of bananas as a sign of the great waste and decadence of Pyotr Shuvalov, a chief advisor to Elizaveta Petrovna:

a most delicious dessert...[which the guests had not heard of, including] bananas in abundance

Here, bananas would not be a financial possibility to the Jewish middle-classes, though nobility may have imported them on occasion.

Bananas were grown in greenhouses at Tolstoy's estate, and had been grown in Britain as early as 1830; throughout the final decades of the nineteenth century bananas began to appear more in places like Moscow, Petersburg, or Sevastopol, but it seems like they would still have been quite expensive--perhaps for a special occasion, in an urban area, a middle class Russian could purchase a banana. I think it is far more likely he would have had one once he was in Chicago, though.

The rise of the banana as an accessible, popular fruit highlights many of the dramatic changes in globalization, trade, and economics during your grandfather's lifetime. Bolstered by American interventions in Latin America, bananas became widespread during the decades your family immigrated to Chicago; by 1910, bananas were found throughout the United States, and were seemingly regarded as affordable. While bananas have short shelf-lives, they grow readily and require less labor than many fruits--they were cheap (relatively) to bring to Chicago, and once there would last only a short while, so prices were pulled down. A 1910 recipe for sliced bananas in the Chicago Tribune opens "Bananas in the house are often...a convenience, almost too convenient" (1910); a Sunday edition from 1908 has a large, ornate advertisement for bananas, differentiating competitors on quality and price. Apples, for example, are grown locally to Chicago, but require greater time to grow, a greater labor investment in picking them, and can be stored for a great while.

A poem by T.S. Eliot, "Sweeney among the Nightingales," casts bananas as accessible and normal amongst other fruits, though the recipient is decadent; by the next decade bananas are a recurrent theme in Faulkner's As I Lay Dying, set in a poor, rural county in Mississippi and punctuating a scene with their bleak absurdity. How bananas were perceived throughout the early twentieth century is more complicated (they're cheap and accessible, but that affordability rests on military invention and labor exploitation; and it looks like their earlier association of excess never quite left them), but by 1915 or so, bananas would quite likely be in the diet of your ancestor.

I can't speak to Russia, but on the United States side of things, the government at the turn of the 20th century recognized the importance of diversifying American agriculture. Too much emphasis on single crops and single varieties of those crops could have disastrous consequences for the food supply if environmental conditions or pests wiped out the year's harvest. Agricultural diversity was a serious goal. Finding crops that were drought, cold, heat, or pest resistant could help secure the food supply and expand where these crops could be gown geographically. The USDA began sending botanists around the world to gather all manner of crops and plants that could be useful to the country.

David Fairchild was one of the botanists who traveled the world (proverbially) gathering every seed he could get his hands on, including numerous varieties of mangos, citrus fruits, hops, etc. We also owe kale and quinoa salads and avocado toast to Fairchild (though perhaps millennials should take some credit for making them popular). He even had a hand in introducing the Japaneses cherry blossom trees that now line the National Mall in Washington, D.C.

Fairchild advocated for new plants and plant varieties to be introduced in a methodical, scientific way. In 1898, the government took his advice and established the "Office of Seed and Plant Introduction" within the USDA. Fairchild was the first head of the office. He was responsible for sending out "explorers" to gather specimens, testing the plants, and distributing them to farmers. The stories of these explorers are really quite wild—especially when trying to get samples of plants that were of local or national pride. Bavarian hops and Corsican citron both involved some "beg, borrow, steal" tactics.

Since bananas have been covered, as a shorter answer to your follow-up question, it's possible he tried mango. Fairchild and the other explorers had a particular fascination with finding mango varieties that would grow well in Florida. But mangos were first introduced to Florida decades prior in 1833, so if your great-great grandfather had an adventurous palate, he may very well have tried mango.

Sources:

The Food Explorer: The True Adventures of the Globe-Trotting Botanist Who Transformed What America Eats by Daniel Stone. (A fun read about the life of David Fairchild and the work of the Office of Seed and Plant Introduction)

"The USDA Plant Introduction Program" chapter from the ebook Crop Wild Relatives and their Use in Plant Breeding by Karen Williams and Gayle Volk

The World Was My Garden by David Fairchild

"History of US Plant Introduction" by Howard L. Hyland