So I wrote this most of this up already for that post but somehow I couldn't find it again, but I guess I'll put it here instead. I will mention though that I don't know as much about modern Japan following Matthew Perry, so I'll leave that to someone else to cover.

So interestingly enough, Japan, being in the same area of east Asia, received significant influence from Chinese military ideologies and thoughts, however the simple fact that Japan is an island means that military strategy and doctrine eventually took a different path in comparison to that of mainland China. To explore the evolution of Japan’s military, it’s important not only to examine the basic facts surrounding the changes in tactics and equipment, but also the underlying political and cultural contexts (which to an extent includes religious values) and the associated intrigues that ran parallel to their change.

During the 6th to 9th centuries (as well as before and beyond, though most prominently during these times), a large influx of Chinese culture entered Japan (including Confucian writings on philosophy, family, law, politics, alongside Buddhist writings, as well as a large number of texts documenting the Chinese ideals surrounding the nature of war – a notable piece would be Sun Tzu’s Art of War), essentially creating the foundation of concepts on warfare that would persist in the Imperial Court. Chinese military doctrine, according to Karl Friday in the book Samurai, Warfare, and the State in Early Modern Japan, has the fundamentals of Chinese warfare pinned as a:

political theory cast the governing institutions of the state as nothing more – and nothing less – than a conduit for the expression of the will of the sovereign…Thus war, in the Chinese scheme of things, could be pursued only by the rightful sovereign, and only if conducted as a matter of last resort…The success of any military venture was in itself proof that the campaign had been in accord with the cosmic order, and therefore by definition right and just.

This idea that the Emperor was not only the Divine Representative of Heaven on Earth but, as an extent of this aforementioned role, the supreme military authority, meant that the mobilisation and campaigns that involved large numbers of soldiers required the approval of the Imperial Court (approvals which took long and complex processes of near unanimous approval by the ruling class).

Also, because of this same line of thought, that of no definitive border between foreign and domestic conflict, you’ll find that the defense required by the state and capital, as well as that same defense (and policing) of everyone else, was undertaken by the same people. I’m hesitant to mention the word class, since really this kind of social structure did not exist (in the same sense that lords, vassals, aristocrats and the like were proper ‘classes’ in Europe) during these early stages of Japanese civilisation, however, but nevertheless – because of the Emperor’s role as a military leader and the Chinese-based warfare doctrine, in turn you’ll find that many armies of this era (that is, largely before the 12th century) were small in number (very rarely over a thousand, often less than a hundred).

Also, because of this, ‘wars’ were less ‘wars’ in the modern definition and more small skirmishes. In this same era, the bow and arrow alongside riding horseback was the primary way of fighting, and as such this also contributed to the minimal number of professional warriors – the price and maintenance of your horse (actually, it’s also worth noting here that the horses used were not the same horses that dominated the European landscape – these horses were no cataphracts nor winged hussars, but smaller and stockier in build, closer to an average or above-average sized pony than a warhorse, and as such for most of Japanese history cavalry fit the role of skirmishers rather than elite shock troops), as well as the skill and time it took to master the bow, meant that the elite class were really the only class of warriors.

Now would probably be a good time as well to give an overview of how these battles would progress. Warfare was largely relegated to the individual, where for the most part, fighters would fire arrows at each other, pairing off and riding past each other firing arrows at each other. Spears and polearms were also in favour, often used as sidearms or as a tool to knock the other rider off his horse, and to a significantly lesser extent, swords (another major caveat that I should also add in to this – katana were not a universal weapon used by anyone. Spears or bows were much better).

However, these small armies were largely ineffective when dealing with larger armies such as the Emishi of northern Honshu - this eventually lead to the creation of a separate military entity alongside the Emperor (or rather, appointed by the latter) – known commonly as samurai and more accurately as bushi, these warriors (which were the same warriors as described before) became the Imperial State’s “claws and teeth,” and similarly these privately trained warriors were bestowed appropriate titles to justify their use of force.

Another massive caveat – the samurai didn’t suddenly transition into the supreme warrior class that dominated the battlefield and the political scene, but rather slowly encroached upon Imperial domain. During the Heian period (8th to 12th centuries), the justification of warfare undertaken by bushi still remained solely within the hands of the imperial state, and was maintained as such despite the establishment of the supreme military general (or shogun) across the Kamakura and Muromachi shogunates.

However, as I mentioned in my opening statement, the political contexts that ran underneath all of this development is key to the shift in military doctrine – families such as the Fujiwara and Hojo controlled various roles such as the next imperial regent, and to a further extent, the appointment of the Shogun. During the successive shogunates however, the Shogun managed to often influence the Emperor’s succession and rule, in some cases forcing the abdication of the current Emperor in favour of a nephew or grandson more sympathetic to the current Shogun’s wishes.

This is what I meant by a slow encroachment by the military class – the Genpei War, fought between the Taira and Minamoto families (Heike and Genji), both who had influence over Imperial succession (see what I mean by the political context?) vied for power, eventually culminating in the establishment of the Kamakura shogunate as well as the initial stripping of power from the Emperor as a military figure. Also, to give you some more of a feel of how complex the political issues were between families that controlled the succession of shogun and Emperor, Jeremy Black's chapter in War in the Early Modern World summarises a later period, the start of the Sengoku era (or more, the Onin War), as thus:

Of the three leading constable families in the service of the Ashikaga—the Hosokawa, Hatakeyama, and Shiba—two, the Hatakeyama and Shiba, were by mid-[15th]century divided by succession disputes that had degenerated into open warfare. Lacking a son to follow him, [the shogun Ashikaga Yoshimasa] persuaded his brother to stand in line to become the next shogun. Meanwhile, in 1465, Yoshimasa’s wife gave birth to a boy, and she immediately formed a faction backing this son, instead of Yoshimasa’s brother, to be the shogunal successor. The two most powerful constables at that time, Hosokawa Katsumoto and Yamana Sozen, each seeking to outdo the other in their rivalry for power, began taking sides not only in the Ashikaga dispute but also in those of the Hatakeyama and Shiba.

This slow, gradual process relegated the Emperor,the Divine authority, to a cultural and ceremonial role (another large caveat, sorry – just because the Emperor no longer had military control didn’t mean that he was somehow insignificant. The Imperial Court still had the final ‘say’ in appointing the Shogun, and Shoguns throughout Japanese history still attempted to gain recognition and validation from the Imperial State. The Emperor’s cultural role was absolutely huge in terms of this, and not to be discounted) whilst the Shogun transformed from the military arm of the Imperial State to that of a Supreme Military General (first letter capitalised for effect).

To build upon that caveat - the Shogun's power came not only from his (and his clan's) military strength, but also from the legitimisation of his role by law (from the mouth of the Imperial Court and the Emperor) - indeed, the original and simplistic idea of a supreme military general came forth out of a direct need for a centralisation of the scattered military powers of different families. It's hard to emphasise exactly how much cultural significance the Imperial Court had - it was enormous, as it was the solidarity and culmination of hundreds of years of religious and cultural ideologies that allowed the Emperor to be a figure of divine authority. Whilst there was a more concrete fusing of the Court and the Shogun during the Tokugawa shogunate/Edo period, the Emperor was still an individual separate entity to the Shogun, and retained his religious and cultural value.