The root of that difference and most of the way both slave societies handled their race relations have to do with the way the land was settled. In the US, colonists frequently traveled with their families from England, and handled land disputes with the natives through treaties and a clear demarcation of borders between the societies.

In Brazil, by contrast, the overwhelming majority of colonists from Portugal were single males who came to seek riches or as a sentence for a crime. That means the first generation of colonists born in Brazil were mixed race people from the local tribes and the Portuguese. The way land claims worked in Portuguese America, the Pope having granted that land to the king, meant all natives were sitting on rightful crown lands. That resulted in a more blunt intrusion into indigenous lands and assimilation of the natives through marriages between the male Portuguese and native women. That pattern repeated with the transatlantic slave trade, with most Portuguese colonist still being male and often having children with their African slaves, witch by law set the mother free and made the children freeborn.

That means most people in Brazil were mixed race, including the elites. That still left a massive economic valley between wealthy white and mixed people agains the overwhelmingly poor and dispossessed black population, that created a unofficial division in the living space of both groups. And although there were no laws that targeted race, there were laws banning traditional Afro-Brazilian customs like Capoeira and Samba.

So, TL:DR race mixing in Brazil was too widespread to adopt a system like the US that neatly divided people into boxes under a one drop rule, but there were other ways implemented to keep the black and culturally African population separated from the westernized elites of various ethnic backgrounds.