So swing, big band and jazz all have very similar roots, and we will use the short hand of jazz to cover for all three.

Blues, rock and roll, metal, any popular music that we know today had evolved from the mash of slave songs and African American spiritual music mixing with classical music, slowly developing and branching out over time. Many genres hold similarities, or are subcategories of another, but it's not uncommon for a genre to have been associated with the Devil, or other "evil" concepts, generally by the stereotypes of the genre's following. Metal heads, techno artists, and many others have experienced prejudice from those who find the sounds a little... less pleasing.

When jazz exploded out of New Orleans in the 1910-1920 range, this was a period in which segregation was still a very popular theme, and the general populace was encouraged to listen to "their music", or that of their own race. As jazz and blues became more popular, and even emulated by Caucasian artists of the time, the sounds spread further and became more accepted by (mostly) younger populations of different ethnicities.

As you've probably seen even in our generation, sometimes those with a more conservative or hesitant viewpoint were quick to speak out, especially as they saw the youth's wild swing dancing, a stark contrast to the waltzes that preceded this era, calling names and threatening, and the age old "I'll show you REAL music!".

The insult of Devil's music was, in this particular case, a racist ideology, set to try to adhere to old beliefs.

Of course, there is also to note that jazz often used different chord voicings and rhythmic patterns than what the popular say was in the 1910s, and these somewhat dissonant, crunchy chords could leave you on edge. Jazz uses "leading tones" that the ear recognizes and expects a certain sound to follow, and the delay or sometimes complete lack of gratification of hearing the resolving chord could also contribute to it's association with the devil.

Hope this helps, apologies for the lack of sources- the information here is as in my notes for my most recent music history course in my degree, so I'm sorry if I took shite notes.

I'm going to point you to a previous answer I made on a similar topic, feel free to ask any questions if you want more information: https://www.reddit.com/r/AskHistorians/comments/3x3u31/in_recent_history_rap_rock_and_roll_and_jazz_were/cy1lj5p/

Can you give an example of the kind of association that you're thinking about?