If you are looking for legal opinions, there are a large number of Court cases throughout US history on race - notably under citizenship laws (for much of US history only "free white" people could become naturalised citizens - later expanded to cover "persons of African nativity or persons of African descent").

There are two Supreme Court cases I think are particularly relevant here.

Ozawa v US 260 U.S. 178 (1922) concerned a man born in Japan but who had lived in the US for 20 years. He sought citizenship but was rejected due to not being a "free white" person.

The Supreme Court was unanimous in rejecting Ozawa's appeal, with Justice George Sutherland writing the opinion. Most relevant toy our question is this paragraph:

Manifestly the test afforded by the mere color of the skin of each individual is impracticable, as that differs greatly among persons of the same race, even among Anglo-Saxons, ranging by imperceptible gradations from the fair blond to the swarthy brunette, the latter being darker than many of the lighter hued persons of the brown or yellow races. Hence to adopt the color test alone would result in a confused overlapping of races and a gradual merging of one into the other, without any practical line of separation.

He continued:

The determination that the words 'white person' are synonymous with the words 'a person of the Caucasian race' simplifies the problem, although it does not entirely dispose of it. Controversies have arisen and will no doubt arise again in respect of the proper classification of individuals in border line cases.

So the Court is saying that it isn't skin colour that matters, but ancestry and origin. However he also notes that this will cause problems with the border cases, but effectively shrugs and leaves that to the "the gradual process of judicial inclusion and exclusion" - i.e. individual courts ruling on individual cases.

The other big case is US v Thind 261 U.S. 204 (1923). Bhagat Singh Thind was an Indian Sikh who identified as "high caste" and claimed to be Aryan/Caucasian. In this case Thind was initially granted citizenship, but the Government appealed. Again Justice Sutherland gave the opinion.

The problem for the Court is that if they say "white" is not a test of colour (which would be impractical for the reasons given in Ozawa) but is instead one of ancestry (so "Caucasian"), Thind's petition would be successful as he could argue Caucasian ancestry. They need some way to get around this - from the opinion:

Mere ability on the part of an applicant for naturalization to establish a line of descent from a Caucasian ancestor will not ipso facto to and necessarily conclude the inquiry. 'Caucasian' is a conventional word of much flexibility, as a study of the literature dealing with racial questions will disclose, and while it and the words 'white persons' are treated as synonymous for the purposes of that case, they are not of identical meaning-idem per idem.

So the colour test doesn't work (it would let in "pale" people they didn't want, while denying "darker-skinned" people they did want), but neither does an ancestry test (which would let in people they didn't want).

The Court gets around this by pointing out that "Caucasian" is a scientific term (or was at the time), and would have been unfamiliar to those who drafted the legislation in the 1790s (although in Ozawa they dismissed arguments as to what those who drafted the legislation would have been thinking about). Instead they note the law uses the term "white persons" and that "these are words of common speech and not of scientific origin".

in this country, during the last half century especially, the word ["Caucasian"] by common usage has acquired a popular meaning, not clearly defined to be sure, but sufficiently so to enable us to say that its popular as distinguished from its scientific application is of appreciably narrower scope. It is in the popular sense of the word, therefore, that we employ is as an aid to the construction of the statute, for it would be obviously illogical to convert words of common speech used in a statute into words of scientific terminology when neither the latter nor the science for whose purposes they were coined was within the contemplation of the framers of the statute or of the people for whom it was framed. The words of the statute are to be interpreted in accordance with the understanding of the common man from whose vocabulary they were taken...

So rather than using any scientific definition or test, they will use the test of the "common man". The opinion continues:

It may be true that the blond Scandinavian and the brown Hindu have a common ancestor in the dim reaches of antiquity, but the average man knows perfectly well that there are unmistakable and profound differences between them to-day; and it is not impossible, if that common ancestor could be materialized in the flesh, we should discover that he was himself sufficiently differentiated from both of his descendants to preclude his racial classification with either.

The Court goes into some of the science behind racial classification (it dismisses the terms "Aryan" and even "Caucasian" to some extent, and comes close to the modern idea of racial classification not being particularly sensible) but decides that it is not helpful:

It does not seem necessary to pursue the matter of scientific classification further... The words of familiar speech, which were used by the original framers of the law, were intended to include only the type of man whom they knew as white.

I.e. forget science, forget colour, instead we are going with "who normal people think of as white", based on history and culture. The Court concludes:

What we now hold is that the words 'free white persons' are words of common speech, to be interpreted in accordance with the understanding of the common man, synonymous with the word 'Caucasian' only as that word is popularly understood. As so understood and used, whatever may be the speculations of the ethnologist, it does not include the body of people to whom the appellee belongs. It is a matter of familiar observation and knowledge that the physical group characteristics of the Hindus render them readily distinguishable from the various groups of persons in this country commonly recognized as white. The children of English, French, German, Italian, Scandinavian, and other European parentage, quickly merge into the mass of our population and lose the distinctive hallmarks of their European origin. On the other hand, it cannot be doubted that the children born in this country of Hindu parents would retain indefinitely the clear evidence of their ancestry.

So... they leave us with a very general test. Does the common man think this person is "white" based on observation and knowledge? Not just based on science, or ancestry, or colour, but based on all these things, combined with culture, history and context.

And that is reflected in other case law, where various groups of people could be classified as "white" by one court but not by another (e.g. a 1908 case on whether Finnish people counted as "white", or the disagreement in defining Parsi people, classified as "white" in a 1910 Court of Appeals case, but 30 years later that they weren't, or a 1915 case which defined "Arabs" as "white", but which was interpreted narrowly as only covering Syrian Arabs until broadened in a 1944 case).