I'm prompted to this question because I have recently seen some single-sex organizations argue in favor of staying that way by invoking notions of tradition and the constitutional right to free association. I have half a memory of reading of these same arguments being made for segregated communities and restricted country clubs, but it's only half a memory. Could an expert weigh in on the counterarguments to integration in the 1950s-1970s or so?
One (narrow) argument in favor of racial segregation comes from where integration was handled poorly.
Adelaide Cromwell wrote The Other Brahmins: Boston's Black Upper Class, 1750-1950 and is still with us at age 99. She got her PhD from Radcliffe in 1946. After a long and fascinating career, being the first African American tenure track professor at a few universities before settling in at Boston College, she was persuaded to publish this book which is a very slight reworking of her doctoral thesis.
Getting to your question, part of the book addresses challenges in the career of black doctors in the 1940's. Boston hospitals were integrated. In practical terms, the black and white patients may be in the same ward and get, one hopes, similar care. However, the medical staff were generally all white. Hospitals in the deep South were sharply segregated with the one small advantage that they set aside a few positions for black doctors and nurses. A black doctor at the beginning of his career, who had find/serve his internship before being a proper qualified doctor had to leave town and do his tour of duty in the deep South. If they had deep family ties in Boston, they may return to private practice there, serving black patients.
A generalization could be drawn about the narrow group of black professionals who had managerial responsibility, e.g. school principals. Almost no school district that had recently integrated would tolerate a situation where white women teachers are subordinate to a black principal. The black professionals were subject to disproportionate demotion and firing...still, more an argument in favor of ethical and competent integration rather than segregation.