Oda Nobunagas legacy is influenced a lot by his struggles against the fringe elements of Japanese society, the Tendai sect Buddhist warrior monks on Mount Hiei, and the shinobi (original ninjas if you will) of Iga and Koga provinces. Buddhism played a big role in Japan in the Tokugawa period that followed the age fo Nobunaga immediately, as the Tokugawa shoguns used the religion as a mean of making censuses and keeping check on the population. This gave an elevated position of Buddhist monks in Japanese society.
The act that really defined the "evil" part of Nobunagas rule is his destruction of the Enryaku-ji Temple on mount Hiei in 1571, which included the massacre of 3.000 men, women and children, in his effort to destroy the warrior sects. Nobunaga came in direct conflict with what was at that point the most influential religion in Japanese society, and even gave his support to Christian missionaries in Kyoto just to affront Buddhism.
It was this opposition to the Buddhist sects that were influential after 1600, int he time when the history of Nobunaga was mostly written, that to a large decree determined his legacy as a brutal and excessive ruler, when the truth probably is he wasn't dramatically more brutal than other Sengoku daymio.