The answer really depends on the time and place you're interested in and to what extent they had developed some form of musical notation. It’s a very complicated subject that involves many different disciplines - from music to physics to archaeology and linguistics. There is a lot of guesswork and judgement calls that go into reconstructing ancient music, and when you hear something like this, I can guarantee it is at least as much interpretation as it is historically grounded. The short answer is that it's impossible to truly know what ancient music sounded like.

Keep in mind also that the concept of what we consider to be music might not apply in certain contexts. Where, how, and why sounds are made vary a great deal throughout human history, and, for instance, a talking drum or a hunting horn serve very different purposes from instruments designed for performance - and those only are forms that survive today.

In terms of figuring out how the instruments were played, you’re often working from the shape of the instruments and possibly surviving drawings of musicians. You can get a lot out of building a reconstruction of one of these instruments and just improvising on it. Contemporary musicians are physically identical to ancient musicians, and figuring out the best way to play an ancient instrument can just come down to trying it out and seeing what works.

Another way of determining what people might have played and how is through reverse engineering from surviving music forms, by looking at contemporary folk traditions. This is problematic, because folk traditions of course change over time. There is an entire field of study dedicated to determining how these traditions developed, and sometimes they can trace back specific changes - for instance, if a particular rhythmic form was brought over during an invasion. Sometimes it's not so clear. If we’re talking reconstruction, a lot of the time contemporary performance is all you have to go on to fill in gaps.

You can break a song down into two basic components: melody and rhythm. Melody is the easier of the two to reconstruct. There are a few reasons for this, starting with simple physics. Without going into too much detail, music is created by vibration. With a string instrument, it's the vibration of the string, with a horn instrument, it's the vibration of the air through the horn. If you have two strings that are exactly the same length and you pluck them both, they will vibrate at the same frequency and produce the same note. Now if you take a string that is exactly half the length of the other string, it will produce a much higher pitch but still sound 'consonant' - in harmony with the other string. That is because you create an octave by halving the period of the sound wavelength, a ratio of 1:2. Now if you take a string that is 2/3 the length of the first string, it will also sound consonant and produce a perfect fifth. You can continue on like this, cutting strings to different lengths to try to find what harmonizes with what you've got. You'll also notice that you can't stack ratios of 2:3 to reach a ratio of 1:2, so perfect tuning is physically impossible. How exactly people choose to resolve that discrepancy has changed a lot over time (and been the subject of some very nasty debates). Here's something more in depth if you're interested. This is all to say that you can determine something about what notes people used by looking at their instruments, because of physics.

Some notes are much easier than others to 'discover' by cutting strings into different lengths, and those tend to show up more often in different musical cultures. The pentatonic scale shows up a lot, because the notes in it are all easy to ‘discover’ by experimenting with string lengths.

Sometimes you’ll have a written record to work from. Melodic notation was developed much earlier than rhythmic notation so you might have some music theory to work with, depending on which culture and how early we’re talking. Pythagoras is typically considered the father of music theory in the West, so we’re talking about the 6th century BC that you start to see more comprehensive systems. More recently we’ve discovered some cuneiform tablets that outline Mesopotamian music theory, which date to around 1500 BC. This can give you an idea of what scales people used, among other things.

Your biggest problem in reconstructing ancient music is figuring out the rhythm. We didn’t develop rhythmic notation until relatively recently - it's actually a really complicated process when you think about it, trying to accurately write down a complex rhythm. Even medieval music notation lacks proper rhythmic indications. There are a few different strategies people use to reconstruct ancient rhythms:

- You can try to reverse engineer the rhythm from lyrics. Poetry has rhythms similar to music, with strong and weak syllables. This is a relatively reliable way to reconstruct a rhythm.

- You can reverse engineer from the closest extant notation. This assumes that you have something close-ish to work from. There are common rhythms that are shared by many songs (we have our own too).

- You can reverse engineer it from contemporary folk traditions. As mentioned earlier, this is a pretty rough way to go, but if you don’t have anything else to work with and you want to reconstruct a song, it might be your best bet.

- The music might just well be in free time, meaning it has no discernable beat at all and was freely improvised.

Oh, the other thing is improvisation. Keep in mind that we can only preserve what was written down, and it was only in the 20th century that we developed the technology to preserve improvised music. We know that a lot of medieval music was improvised, but after the development of better notation, this tradition died out because it couldn't be recorded. Reconstructing ancient improvisation is mostly down to the instruments themselves and a lot of judgement calls.

This is just brushing the surface, really, this is a massive field of study, but I hope it gives you a decent overview.