Well this is a hard question to answer. It's difficult to know if the places represented in Marvelous Mrs. Maisel (MMM) are representative of New York at large in 1950's, or merely of the underground hip comedy scene in which most of the show takes place. You also have to reconcile the large and torrid history of New York racial relations which is.....not good to say the least.
First let us focus on New York as a whole. Being part of the North, states like New York are often "let off the hook" when it comes to segregation or Jim Crow. But as Malcolm X writes in his autobiography in 1965 “Ultraliberal New York had more integration problems than Mississippi," and "The North’s liberals have been for so long pointing accusing fingers at the South and getting away with it that they have fits when they are exposed as the world’s worst hypocrites.” X goes on to describe New York's (and the rest of the norths) hidden racism and their penchant for making de facto segregation despite lacking official anti-miscegenation or outright "Jim Crow" laws.^(1)
For example Dorsey v Stuyvesant Town Corp is a court case heard by the New York supreme court in 1949, in which Stuyvesant Town, a housing development championed by Mayor LaGuardia and Parks Director Robert Moses^(2), was sued by two black WW2 veterans for discriminating against renting to black Americans. According to the court ruling "The State of New York has consciously and deliberately refrained from imposing any requirement of nondiscrimination upon respondents as a condition to the granting of aid in the rehabilitation of substandard areas. "^(3) in other words, sorry guys, they don't have to rent to you if you're black.
Accordingly while New York had a public housing law against discriminating in purely state funded works^(4) there was no sweeping anti-discrimination law for private landlords or developments outright until the Fair Housing Act in 1968.^(5)
Therefore we can say that the cast of MMM would be rather unlikely to encounter any African American neighbors in their housing.
But what about restaurants? In 1950 the Committee of Civil Rights in Metropolitan New York incorporated to explore segregation in new york^(6). They essentially went around to restaurants with African American diners and white "control" diners and rated the differences in their service. In 1950 the CCRM found evidence of discrimination 42% of cases. Though no diners were outright refused service they often got poorer service, less preferential tables, and was more pronounced in higher price establishments.^(7) The CCRM also interviewed a man who provided the following quote. "Every time I’m downtown I see invisible signs on the door of every restaurant, saying “Negroes, Keep out!” I make up my mind to have a good meal, and I walk for blocks, looking at every eating place I see, and wondering...And then I get cold feet and end up at a cafeteria - or else get a sandwich in a drug store."
What we can infer is that while there was no "official" segregation at the time African-Americans faced societal and unspoken pressure to segregate.
You can also look up information on the fight to integrate new york schools which largely didn't happen till the mid 60's.
So to answer your question, "was New York that integrated at the time" yes and no. It wasn't "as bad" as the Jim Crow south but it certainly wasn't completely integrated. However given that MMM is not centered around African American characters it likely is technically accurate if not a little generous in regards to frequency of their appearance.
Sources:
X, Malcolm, and Alex Haley. The Autobiography of Malcolm X. New York: Grove Press, 1965. Print. ^(1)
Caro, Robert A. The Power Broker : Robert Moses and the Fall of New York. New York :Knopf, 1974. ^(2)
Dorsey v. Stuyvesant Town Corp., 299 N.Y. 512, 528 (N.Y. 1949) ^(3 4)
United States. Civil Rights Acts of 1957, 1960, 1964, 1968, and Voting Rights Act of 1965. Washington :U.S. Govt. Print. Off., 1969. ^(5)
Committee on Civil Rights in Metropolitan New York, Inc. records, 1950-1966. Amistad Research Center. ^(http://amistadresearchcenter.tulane.edu/archon/?p=collections/findingaid&id=117&q=&rootcontentid=66627#id66627) ^(6 7)