Two books are:

• Michael B. Poliakoff, Combat Sports in the Ancient World: Competition, Violence, and Culture, Yale University Press, 1987. As the title suggests, this is restricted to sports (mostly wrestling, boxing, and pankration), and focusses on Greek sports. Egyptian stick-fighting is covered in a few pages.

• Peter A. Lorge, Chinese Martial Arts: From Antiquity to the Twenty-First Century, Cambridge University Press, 2012. Covers ancient, Medieval, and modern Chinese martial arts, with about 1/4 of the book covering ancient China up to the end of the Han Dynasty.

The most important and prominent ancient Chinese martial art was archery, and a significant amount of written work on technique, practice, and philosophy has survived. A useful book collecting material on Chinese archery, ancient and later, is:

• Stephen Selby, Chinese Archery, Hong Kong University Press, 2000

Moving past "ancient", much more material has survived from the late Medieval period and later, from Europe, the Middle East, China, Japan, and Korea (I'm not familiar with Indian martial arts texts). The majority of this material deals with armed martial arts. Some deals with unarmed martial arts, and some is heavy on philosophy/theory. One example of the latter, from the 18th century, is

• Chang Naizhou (original author), Marnix Wells (translator and editor), Scholar Boxer: Chang Naizhou's Theory of Internal Martial Arts and the Evolution of Taijiquan, North Atlantic Books, 2005