I assume it entered the larger consciousness due to slapstick vaudeville shows and early movies in the early 1900s, but how much earlier than that does the idea of someone slipping on a banana peel go?
They don't become popular imports until then. Dan Koeppel ( Banana:The Fruit that Changed the World) states that the early variety of imported bananas also had a very tough skin, and that they were much more slippery therefore than the present thin-skinned Cavendish ( which doesn't present so much of hazard).