It seems to have been sometime in the mid-1960s that American popular culture first became aware of the concept of the black-clad elite warriors. In 1964, Ian Fleming published You Only Live Twice, which prominently features a force of ninja warriors and spies. This was deeply exotic at the time, and the depiction in the movie reflect the deeply foreign nature of the concept. This is acutally cited by the Oxford English Dictionary as one of the first appearences of the word "ninja" in English outside of specialist literature on East Asian history. That same year, the summer 1964 Summer Olympics were held in Tokyo -- this was the first time that many Americans saw genuine Asian-style martial arts on televsion, and overall helped lead to the ongoing cross-pollionation that characterizes excahnges between American and Japanese popular culture to this day.
Also worth mentioning is influence of the 1980 minseries Shogun, a sprawling epic set in 17th-century Japan. Although it's hardly a work of scholarship, it's the closest to a historically-accurate vision of the ninija that most Americans have any reference point with.
You may find this article edifying and entertaining, although much of it break the 20-year rule: http://io9.com/5982187/why-americans-became-obsessed-with-ninjas
And for other countries, the concept of the ninja entered Australian popular culture in 1962 when the black and white television series The Samarai premiered. It was a series called in Japan, Onmitsu Kenshi, poorly dubbed into English featuring a Samarai hero and Ninja sidekick, with all the villains being either ninja or employing ninja. It ran continuously until 1965, during children's prime time and was hugely popular.