I don't have much to cite but in Western traditions it's very often an allusion to the Bible. In Genesis, Adam and Eve committed the original sin by eating the fruit from the tree of knowledge, so artists will often use fruit to allude to sinful acts.

In some renditions, Eve isn't simply tricked by the snake into eating the fruit but rather is tempted into eating it (and subsequently tempts Adam), and its this aspect of temptation that artists will use to incorporate elements of lust in their works.

Even if they're moral or pious, the artist or the audience still understand that feeling of temptation, so artists would often use fruit because it makes for an evocative lustful image.

There is also a historical quirk that the Church was (is?) very strict when it came to obscene images, such as sinful and lustful behaviour, and would often censor artists. If it was an images that depict events in the Bible, however, it might be slightly more acceptable, so it became the go-to imagery when an artist was feeling a bit saucy (i.e. wanting to discuss the feeling of temptation and nothing more).

This rigidity also meant that the imagery became embedded into artistic practices (i.e. not specific to the Christian faith), which is why it has meaning to non-Christians as well. The continued use of the imagery further perpetuates the meaning, and artists (Christians or not) continue to latch onto it as easy shorthand.

And, I should note: Fruit may have lustful or sinful connotations for other cultures as well, this is just one perspective, and I can't speak to those.. either.