How much does the typical image of a Japanese ninja align with how they really looked/operated?

Nowadays the popular image of ninja is generally depicted as a person in black clothing. This particular stereotype, however, is widely known to be fiction. The clothing originates from the so called 黒子 kuroko (black person) who operate as stage assistants in kabuki theater, a form of theater since the 17th century for the common folk. This technique was also used in Nô theater, which was a more formal theater for samurai, and bunraku theater, which is a puppet theater. ^1

The kuroko were people dressed in black, they usually moved set pieces around or played inanimate objects. They were supposed to be seen as little as possible. They even hid behind actors and matched their breathing in order to be as inconspicuous as possible. The black clothing was to symbolize that they are invisible and thus not part of the play. Later this was extended to depict stealthy ninja that operate in the shadows and surprise the audience. This image later carried over to popular culture all around the world to this day. ^2

Actual ninja actually mostly wore civilian clothes as they acted as spies and participated in espionage. They were active throughout the 9th to 17th century. However, only since the 15th century are we seeing accounts that guerrilla warfare, spying and assassinations were carried out by trained individuals in specific groups. Ninja that primarily operated in the Iga and Koga regions of Japan had two different roots. One being the role of an undercover agent and spy, the other being the role of a mercenary. This was contrary to the image of a samurai, who were supposed to be noble and honorable. Mercenaries, ninja being one of the few examples of mercenaries in Japan period, were not seen as noble, thus the daimyo (leaders of regions of Japan under the shogun in medieval Japan) did not want the image of their samurai tainted and thus they hired highly sought after ninja that operated through specialized organizations. ^3

As mentioned above, they were active as mercenaries and spies, but they also conducted secret raids on castles such as the destruction of the fortress Hachiman-yama and an attack on Ototsu castle both in the 14th century. Or they were brought along as mercenaries, especially during the sengoku jidai (the warring states period from 1467-1603). Interestingly, ninja were often not treated well by their allies since they were often perceived as petty thieves, unless they were brought as mercenaries.

In the mid 15th century, certain samurai families started developing particular skills in espionage and martial arts. These skills were then passed on from the teacher to a student who was chosen by the teacher himself. The student did not have to be a family member.

Thus ninja operated in different groups that were also divided by region, because as of Tokugawa Ieyasu's reign only certain daimyo were allowed to have ninja in their armies at all.

Sources:

^1 Goodman, David, and Kazuko Goodman. Kabuki from the Outside. The Drama Review: TDR, vol. 15, no. 2, 1971, pp. 175–184

^2 Turnbull, Stephen. Ninja AD 1460–1650 (Warrior). Osprey Publishing: 2003

^3 Turnbull, p. 5-7