The idea that there are only 2 genders, and you can tell what gender someone is by looking at their body, isn't a default idea for humans. A lot of cultures had concepts of multiple genders to begin with, and had a more binary system imposed on them by colonization. I don't totally love this resource (some of the language used is pretty outdated) but it can serve as a jumping-off point to quickly learn about some examples of societies with multiple gender roles.