From my limited knowledge it seems like incidents such as these were completely out of step with previous generations and indeed the generations since WW2. What caused this societal change? Did winning the Russo-Japanese war combined with aggressive expansionism give an inflated sense of ego to the Japanese and mob mentality took over?
They're not 'out of step' more than they are the culmination of an expansionist policy that began in the 1890s and bred, as you suggested, a culture of invincibility and aggression. There's also evidence of a racial sense of superiority.
Japanese expansionist policies in WWII were based on several goals, ranging from the immediate strategic concerns to ones based on older ideals and Imperialism. Japanese reasons for beginning their rapid-fire series of invasions across SW Asia at their very basest can be narrowed down to 2 immediate goals:
Strategically, seizing the oil-rich Dutch East Indies, blocking British India (and securing a springboard for potential further action) in Burma and knocking out the forward most US territory in the Philippines.
Out of a twisted sense of nationalism and idealism that saw Japan as the potential centerpiece for a Greater Asiatic empire. This manifested in the rhetoric of "Asia for Asiatics" and the euphemism for a new Empire - East Asian Co-Prosperity Sphere. While its debatable as to when this vision first entered the Japanese conscious (Bill Yenne believes its roots can be found as early as 1890) its very clear that its present before both the Rape of Nanking and the US's innovative and morale-boosting strike with the B-25s.
An excerpt from a pamphlet given to IJA soldiers, roughly translated as "Read this alone: and the War can be Won"
"We Japanese have been born in a country of no mean blessings, and thanks to the august power and influence of His Majesty the Emperor our land has never once, to this day, experienced invasion and occupation by a foreign power. The other peoples of the Far East look with envy upon Japan; they trust and honor the Japanese; and deep in their hearts they are hoping that, with the help of the Japanese people they may themselves achieve independence and happiness."
My emphasis. The rhetoric is telling; and this pamphlet was issued in December 1941 before the Japanese began their lightning campaigns across the Southwest pacific. Its clear that an aggressive sense of superiority was already present in the culture of the Japanese military and that they viewed themselves as superior in position and culture to the peoples they were conquering.
This is made even more ironic by parts of the pamphlet not quoted by Bill Yenne. The entirety of the pamphlet's first Chapter re-iterates the conditions of the campaign area and the mission, stating the importance of the oil and steel resources Japan needs. The second chapter stresses kindness and respect towards the natives - but this clearly was lost in translation for it rarely manifested, the second chapter also details how Asians in this area have been subjugated by whites. This is important for two reasons, first it attempted to paint the Japanese soldier as liberator, and secondly it showed that while Japan had never been "conquered" by European Whites in their own view, these other Asian nations had been. Japan, therefore, was in an inherent position of superiority over these countries.
If this subject interests you, I'd suggest reading Ian Toll's Pacific Crucible. Toll's main story is the first six months of the war in the Pacific, from Pearl Harbor to Midway. His focus is on the naval officers who led those actions.
In explaining those sets of leaders, he describes the history and behavior of the political leaders of both the U.S. and Japan. My take on his description of the Japanese political system is that a relatively small group of military officers and industrialists pushed their government in a hard-right direction.
I think every culture has racist and xenophobic tendencies. The extreme elements in Japan played on those to achieve their goals. (Not that they didn't believe what they were saying. They really did believe in the racial purity of the Japanese people - an opinion perhaps exacerbated and nurtured by the relatively homogenous population of their island.) When they took over other countries, their treatment of the indigenous populations were impacted by their belief that those populations were inferior, a belief that was highly encouraged by the Japanese army.
This belief was not as strong in the navy - possibly because they were exposed to more cultures? Yamamoto, the admiral in charge of the attack on Pearl Harbor, had studied at Harvard, was stationed for a time in D.C. and traveled extensively throughout the U.S.
If you like reading history, I'd highly recommend Toll. This and his other book, Six Frigates are both extensively researched and very well written.