Were all members of the Nazi Party soldiers by definition?

No, they were not, there were soldiers who were not in the NSDAP and NSDAP members who were not soldiers.

At it's hight, the NSDAP (Nazi Party) had around 8 million members. There were over 18 million soldiers in the Wehrmacht, even accounting for deaths, this would not add up. Many members were also judges, teachers or other members of the public and government. Become a member of the NSDAP was actually pretty difficult and could take up to two years. Between 1933 and 1939, the party didn't even accept most new membership applications due to a massive influx after their rise to power. They were cautious about admitting those who were applying to the NSDAP for their own benefit rather than due to believing in the NSDAP.

The easiest example of NSDAP members who were not soldiers is women. As mentioned in the War on the Homefront thread, the Nazis were not keen to women even working in men's jobs and women were not allowed to be soldiers.

Another prominent example that I mentioned above are teachers. The Nazis were pretty keen on teaching their values to children and had a subdivision within the party for teachers (Nationalsozialistischm Lehrerbund). These were actually primarily women too (though of course, not in most leadership positions). There was also a subdivision of civil servants, lawyers and more.

It'd be interesting to see what context the quote you mentioned was from? As discussed in the thread /u/Georgy_K_Zhukov linked to ("Is the use of 'Nazi' as a general demonym for referring to members of the German armed forces during World War II correct?"), the term Nazi is not always used a "member of the NSDAP" but rather as "someone living under or working for the Nazi state". In which case, yes, all soldiers were Nazis but no, not everyone was a soldier.