I'm unaware of Axis soldiers fighting for the Allied Armies during the war, with perhaps the exception of Polish auxiliary troops captured in Normandy, who were recruited to replenish the ranks of 1st Polish Armoured Division.

It is possible that your grandfather may have served in the Italian Royal Army, and then joined the ranks of the Italian Co-Belligerent Army after the 1943 Armistice, and thus fought alongside the Allied Armies in Italy for the rest of the war.