Was Hitler and by extension the Nazis concerned with jews by religion or by race?

This is a complicated question which the Nazis themselves were not fully able to answer. One of the questions which had to be answered in the creation of the Nuremberg Laws was who could be considered a Jew. Traditionally, persecution and hatred of the Jews was anti-Judaism rather than anti-Semitism; however, the late 19th century saw the racial anti-Semitism come to the fore. Hence, the hatred of the Nazis toward the Jews was racial in nature. Nonetheless, this didn't determine who was to be considered a Jew, since, especially in countries like Germany, many Jews had converted.

This is the situation which led to the Nuremberg Laws defining Jews. A short version of these laws is that Jews were those with three or more Jewish grandparents, even if these grandparents had converted. This demonstrates the clear problem...how does one determine if those grandparents were Jews? In other words, the definition of a racial Jew was still based partially on religious identification.

To answer your question from your comment, Raul Hilberg, in The Destruction of the European Jews notes that in a couple of court cases, it was decided that

who is racially a non-Jew but who openly claims membership in the Jewish community, belongs to the community and therefore has placed himself in the ranks of the Jews.

Hence, such a convert would be considered a Jew if he doggedly maintained his/her faith.

Source: Hilberg, Raul. The Destruction of the European Jews. Chicago: Quadrangle Books, 1961.