It's not completely accurate to state that Justices Rehnquist (as he then was), O'Connor and Marshall did not participate in Chevron as Justice O'Connor initially participated but then recused herself.

When Chevron was argued, Justices Rehnquist and Marshall were absent due to health reasons. As a general matter, subsequent to oral argument, the U.S. Supreme Court justices conference to indicate how they would vote on the case. The conference for, inter alia, Chevron was held on March 2, 1984 with Justice O'Connor present. She voted to affirm. By tradition, as the then most junior justice, she would be responsible for getting the door if someone knocked during the conference.

On June 12, 1984, Justices Rehnquist and Marshall circulated memoranda confirming that they should be shown as taking no part in the decision in the case (one could cynically muse that neither wanted to participate in a complex administrative law decision if they did not need to).

On June 14, 1984, Justice O'Connor advised that, after argument, she had inherited a beneficial interest in one of the companies in the case, and she was therefore recusing herself.

Merrill, T.W. "The Story of Chevron: The Making of an Accidental Landmark." Administrative Law Stories, edited by Peter L. Strauss ed., Foundation Press 2006.