There's always more to be said, but while waiting for answers, the first part of your question has a few replies from the music FAQ:

/u/nmitchell076 has a response to "Before the advent of a universal modern music notation, how did people learn and write music?" /u/racecar_ray has a reply to "What did Roman music sound like and what form did the written notation take?"

Before around 900C.E. westerners just used oral tradition with music. One person would teach another a song by singing it. It works very well, and is not especially hard to teach someone a song, but it is time consuming. It was the need to teach music without so much human interaction that spurred on the creation of a written system in the 9th century.

The first real music notation in Europe (not including antiquity) were neumes, which were basically squiggles that generally followed the pitch a performer was to follow. Neumes were used to notate Gregorian Chant, and were written by the church in order to teach new priests all the music they needed to know to run services. This system eventually evolved into our modern Western music notation.

As for other notation systems:

About 100 years after neumes, Solfege was invented in Italy. Solfege uses words to describe pitches relative to one another (or can indicate absolute pitches in modern day), and can be used as a teaching tool in conjunction with Chant notation. This is not truely a notation of it's own as there is no rhythmic component to it.

In China, though, there is Jianpu(what I learned as Simple) notation. This uses numbers to represent pitch, just like Solfege uses words, and lines under the numbers to represent rhythm. This system is more commonly used today compared to China's older Gongche notation. I unfortunately am not familiar enough with that system to describe it.

Also worth noting is Taiko Drumming notation. It uses words as sets of rhythms (just the opposite of Solfege). That way a player can just be told a short series of words and the player will know many notes and can be performed live with one person announcing the next line to play.