So, I'm not a music historian but I have done some fairly extensive studies in epic and lyric poetry as well as both tragedy and comedy, all of which require a great deal of understanding of ancient musical constructions. So I should be able to at least do a decent job, and I'm sure any one of our several music historians running around here will call me out if I make a blunder.
So I'm going to assume we're talking about instrumental music. Not that it really makes much difference, since in the ancient world the concept of music was much broader than ours today. First off, all poetry was intended to be accompanied to music. It wasn't always (for example, Virgil probably wouldn't have accompanied himself on the lyre during recitation of the Aeneid the way a bard singing the Iliad would've) but that at least was the intention. What instrument accompanied sort of depended on the type of poetry. So epic and lyric poetry was usually accompanied by the lyre (although epics could be sung to the aulos as well) and choral interludes were sung to...well, a variety of instruments. The precise types of instruments in the ancient world is quite a complicated study--for example, there are something like a dozen different types of lyre, some of which can be positively identified and others of which cannot (none of them can really be reconstructed). Even when poetry was not accompanied it was still highly musical, because of the meter (this is particularly so in Greek, where the metrical flow of the vowel lengths was synchronized to the pitch accent in a way that produced much more obviously melodic material than the somewhat weaker stress accent of Latin).
With all that out of the way, I'm going to only address the musical evidence based on the fragmentary notation that we have. There is a great deal of evidence for the musical forms long before the date of our first fragment in Greek. The lyric accompaniment of the Homeric Poems, for example, can be guessed at given what evidence and knowledge we have, but it's not very useful for our purposes and is even more speculatory than what we physically have in notation. The cool thing about most Greek music is that since it accompanied various types of verse we can construct to some degree the rhythm of the tune, since it would have to be in some kind of synchronization with the meter and the tonal qualities of Greek. This is an important tool, since most of the fragments that we have have the notation of the melody and the lyrics written, but they don't indicate where the lyrics fit into the melody. Now, what I just said is important--our notation only notates the melody. We have absolutely no idea what any harmonies might have sounded like, and we have no notation for background instrumentation like percussion or anything like that. Nor do we have any idea of tempo (remember this bit, it'll come up later). Now, from what Plato tells us it seems that most Greek music didn't really have polyphony, which makes our lack of harmonies a little bit easier to take, but the instruments most definitely weren't all playing the same thing. This makes reconstruction extremely difficult.
So what did it sound like, more or less? Well in a lot of cases we can guess at the basic "tone" of various types of music, based on what sort of poetry it was supposed to accompany. So, for example, the Homeric Hymns mostly have a particular style and rhythm based on meter and content, and wild, ecstatic meters used in Dionysiac revels would have sounded quite alien to that, being naturally frenzied and harsh. Another thing is that the Greeks categorized their scales, or modes, and thought of them as representing different styles. So, for example, most music written in the Phrygian mode was considered sensual or even erotic, but what exactly that means is unclear. What constitutes erotic-sounding music to the Greeks, and how does it compare to what we'd think of? Those modes were very different from the scales used in most western music. Western scales prefer full tones, and have a system of half tones, occasionally with quarter tones. Greek modes featured full tones, half tones, quarter tones, and various confusing fractional tones which sound bizarre to a western ear.
Now, it used to be said by a lot of scholars that the Greeks liked very orderly music, with single melodies being played to the accompaniment of some rhythmic instrument (like a drum or an aulos) at a nice, even tempo, and evenly-spaced intervals between notes. Most of this is on the basis of what Plato says. But most of what Plato says about music is in the Republic, particularly in passages where he's complaining how lax Athenian culture is and how it's corrupting the morals of the youth. Plato's concept of idealized music is like that, yes, but that's not necessarily the case for everyone else. For example, one of our earliest texts is a fragment of the musical notation in a passage from Euripides' Orestes. The notation quite clearly calls for several notes to be played simultaneously at several points, and the pitches being played don't seem to really follow the Pythagorean rules that scholars had posited previously. It's also unclear how long notes were, since most of the fragments show only the pitch and don't note the duration of the notes or of rests (actually, they don't show rests at all). Again we have to go to our literary evidence. What's of greatest importance here is the actual verse, since we know the duration of the syllables. But that doesn't tell us how the melody synchronized, even though Plato says that lyric poetry should always be in perfect synch with the verse, so we have to sort of guess a bit. We also have descriptions of music. Aristophanes is one of the most important, since he likes to make jokes about the awful arrangements of his competitors and various tragedians as well (mostly Euripides and Agathon). But there's a passage in Plato, in the same passage where he's describing how ideal music should be neat and orderly and follow perfected mathematical patterns, where he complains that some poets these days are performing music that he says sounds "like ants crawling." That's a very interesting passage, one that's puzzled scholars for decades, probably centuries. What the hell does Plato mean by that? More than one scholar has proposed that Plato must be referring to the sensation of his skin crawling, and it's been suggested that this could be some form of atonal music. Plato could also be talking about tempo, which brings me to the last bit. Plato says that the rhythm, which we can probably say means the same thing as tempo, should be even but neither he nor our fragments indicate time. Some poetic forms must have been naturally faster than others--for example, the Dionysiac meters are naturally frenzied and were probably maddeningly fast. But which ones? It's difficult to say, really. Most reconstructions of Greek music, such as the reconstruction of the Seikilos epitaph which another user linked to, are very slow and don't really show a lot of the finery that we see described so much in the textual corpus. They're slow, stately, refined, and highly artificial. This is the main criticism made of them, that they're too artificial. Which is understandable--we don't really know what this stuff sounded like. Many reconstructions, such as the one I'm referring to, seem to base themselves on some musical model that we know about, often early church music. But our descriptions don't often fit into that model, and many of those forms are poorly adapted to Greek--in the reconstruction linked the pronunciation and pitches of Greek had to be greatly altered to fit the music.
So basically, we're not sure. Our Greek fragments preserve the lyrics and the pitches of the notes of the melody, nothing more. Everything else is guesswork. Some things we can guess at better than others, using the literary evidence available and our understanding of Greek meter and pronunciation, but it's not really all that much. Many reconstructions are actually probably very unlike what it was supposed to sound like, because you have to make a great deal of assumptions, most of which are probably wrong
The oldest fragments of music notations are 4000 years old, recorded on cuneiform tablets
Oldest complete surviving musical composition is Seikilos epitaph
Probably not the oldest song that could be reproduced, but a Greek professor of mine (he wasn't Greek - he taught me Greek) sent me a link to this - a reading of the Proem of the Iliad with lyre accompaniment by Stephen G. Daitz, professor Emeritus of Classics at CUNY and honorary president of the Society for the Oral Reading of Greek and Latin Literature. I suppose I can't be certain that it is exactly what the music/recitation sounded like, but Daitz doesn't seem like he would just make stuff up, given his credentials.
http://www.rhapsodes.fll.vt.edu/Iliad/iliad.htm
If this is not an appropriate kind of source for this sub, I apologize!
Well since out of all the people I know who've studied this stuff formally I'm the only one who uses Reddit I guess I'll give it a shot. Forgive my lack of eloquence.
Our ability to reconstruct ancient music is predicated on several different pieces, and types, of evidence. I can only talk about Greek and Roman I'm afraid, I'm vaguely cognizant of some ancient Indic stuff and have attended a few talks on the ancient near East but I retain all to little bar when it is is necessary for reconstructing the Greek material.
We have)
i) Analogical analysis ii) Fragments iii) Texts discussing music theory (musicology) and metrics.
i) E.g. we reconstruct via analogy with poetry, which was quantitative and therefore we're able to get some sense of things. Naturally this is dependant upon our very secure ability to also reconstruct pronunciation and, to some degree, performance models.
Instruments and tunings have a limited range of notes, we're working with a limited range for construction.
ii) Fragments ought to be self evident. We have a number of fragments from a wide variety of style and genres. They don't tell us everything but they're a useful guide. Combined with what we know of quantity and pitch accentuation we're in a good ball park.
iii) Again we have a decent amount of these kinds of works. They tend to be proto-mathematical or aesthetic or, rarely and more valuably, musicological so we have a sense of what each note was in relationship to another, their sense of timing and melody (so different from ours!) and so on and forth.
We're better at reconstruction than you might think and this is a growing area. I'll recommend some books.
West, M L (1992) Ancient Greek Music. Oxford. The industry standard.
Pöhlmann, E and West, M L (2001) Documents of Ancient Greek Music. Oxford - The standard collection of fragments with commentary.
Hagel, S (2009) Ancient Greek Music: A New Technical History. Cambridge. One of my favourite books and invaluable for its more technical standpoint.
Gibson, S (2005) Aristoxenus of Tarentum and the Birth of Musicology. New York. One of the best accounts of Greek musicology ever written.
Everything else I know is much more technical in nature, but that is a fantastic start. Let me know if you have any questions.
While it's assuredly not the oldest music we have access to (as this thread attests), many Christian hymns are quite ancient and still in regular use today. The most famous of these is assuredly Te Deum, which was written in the 4th century. The earliest that we have musical notation for is the Oxyrhynchus Hymn, written sometime in the 3rd century and rediscovered in the 20th.