Oppenheimer was not religious in any sense, as far as we know. He was interested in philosophical issues, and like many quantum physicists (and intellectuals) of his time found it interesting to dabble in eastern philosophies and religion (which seemingly have more tolerance for ambiguity than many Western ones). But beyond that — it seems not. He never did anything that would demonstrate any kind of serious commitment to religion of any sort.

With Oppenheimer in particular, it is hard not to see this as one of several "rebellions" against his upbringing and assumed identity. Oppenheimer was raised a wealthy, secular Jew on the upper-west side of Manhattan, and his father was a hardcore supporter of Felix Adler's Ethical Culture movement. Ethical Culture is essentially a form of secular Jewish ethics — very self-serious, very un-mystical. Oppenheimer's push in two directions (eastern philosophies and "cowboy" culture) both seemed to come with his going to Harvard during a time of extreme anti-Semitic sentiment. So it is easy to paint a portrait of rebellion/rejection there: to embrace Hindu philosophy, and New Mexican horseback riding, are one and the same approach to diversify his sense of self, and to reject his own (and his father's) Jewishness. These sorts of things became hallmarks of his identity, especially in the years after he recovered from a "nervous breakdown" sort of incident in Europe and reinvented himself as "Oppie," the eccentric and confident wunderkind.

This is all interpretation, to be sure. For a very nice psychological portrait of Oppenheimer, I have found Ray Monk's Oppenheimer: A Life Inside the Center very perceptive on these kinds of identity issues.

You may be interested in this answer by u/kieslowskifan and [this] (https://www.reddit.com/r/AskHistorians/comments/36i7q6/what_is_the_significance_of_the_phrasing_i_am/) one by u/restricteddata.