Based on the oldest instruments, we've got 3 contenders:

1 - The Divje Babe flute, dated at 43100 ± 700 years , although there is some question as to whether this is the actual age of the flute or rather of the bone the flute was made of.

2 - The Hohle Fels Flute, dated at about 35,000; that age is generally well accepted

3 - The Geißenklösterle cave flutes (2) may also be 42,000 to 43,000 years.

So that gives you a minimum range.

However, this says nothing about percussive instruments (which could be anything really) and might either be non preserved (wood and leather drums) or about singing. And there is no reason whatsoever to believe that singing/chanting aren't deeply rooted in human prehistory either.

Steven Mithen postulates, in The Singing Neanderthals: The Origins of Music, Language, Mind, and Body, that early hominids likely sang before they spoke. There is some evidence to suggest, as he details, that musical sound precedes linguistic sound in hominid and early human expression and communication.

It's fascinating to see behaviors so common among humans now that have likely nearly always been with us: not just using sound to express emotion and feeling, but aspects foundational to music like communal mimicry or rhythmic movement.

The instruments we've found--cited in /u/Gargatua13013's excellent comment--speak to how fundamental creating and making music ("musicing") is in human behavior, but it is incredibly likely that humans--and earlier hominids--were musicing in various ways long before they figured out how to create tools to make even more specialized expressive sounds, like flutes.

What's difficult about studying music is that it's an activity, a verb. So whenever and however we--or our ancestor organisms--started musicing, it very likely left no physical evidence. (I mean, consider that we've only had complex musical notation for a few centuries, and recording technology for less than two centuries. We have no real record of this ubiquitous human activity until very, very recently, even though it's something we know humans have done for a long, long time.)

Thanks everyone! Very interesting and thoughtful :)