I can't answer whether Latinos were considered "colored" under Jim Crow in the South, however, in CA there were segregated schools for Latino children.

The policy of segregated schools specifically for Mexicans in CA was challenged in the 1940s in the California courts. In 1947, the 9th Circuit Court of Appeals released en banc their decision in Mendez, et al v. Westminster School District of Orange County 161 F.2d 774 (9th Cir. 1947).

While the lawyers for the Plaintiffs argued that the children were "white", the court did not rule on that issue but rather held:

The equal protection of the laws pertaining to the public school system in California is not provided by furnishing in separate schools the same technical facilities, textbooks and courses of instruction to children of Mexican ancestry that are available to the other public school children regardless of their ancestry. A paramount requisite in the American system of public education is social equality. It must be open to all children by unified school association regardless of lineage.

This is the earliest appellate case I'm aware of holding that the equal protection clause of the US constitution forbids segregation of students based on ancestry. While it doesn't answer your question about "white v. colored", it does demonstrate Latinos suffered discrimination due to their national ancestry.

There was also a large Chicano rights movement in California in the 1960s. Someone with more expertise than me can answers questions about that.

I'm very familiar with the legal history of segregation in California so feel free to ask any questions about that topic.