I'm writing a book set in this time/place and want to include as much diversity as possible. Albany isn't set in stone; another town in the southern Catskill-region counties would work, as long as it isn't more than half a day's journey from NYC.
The Hudson Valley region of New York State has one notable ethnic distinction -- descendants of Dutch families. So in New York, and particularly in the Hudson Valley, you get Schuylers, Roosevelts, Rensselaers, DePeysters, Van Alstynes and so on settling early.
By 1890-1900, when my grandfather's family (Russian, Jewish) arrives in Kingston, the demographic forces shaping the Northern US generally were similarly present in Albany and the Hudson. So we're speaking of families of older English, more recent German, Italian and Irish descent. The Mayor of Albany is Thomas Van Alstyne --from an old Dutch family-- in 1898-99, after that you're looking at Cornings, Stevens, McEwen and so on, with just one Dutch name (Hoogkamp) among 20th century mayors.
The Dutch heritage is the unique feature of the region, and by the late 19th century while no one is speaking Dutch, there's still considerable nostalgia and social cachet associated with the patroon families
Getting the Dutch ethnic history of the Hudson Valley is essential to a convincing portrayal of the region's distinct character. No other aspect of Albany, the Catskills or the Hudson Valley's ethnic history as distinctive as this heritage-- indeed, if you look at placenames, you'll find an abundance of Dutch names in New York, and almost nowhere else (Western Michigan is the other State which has a concentrated Dutch settler history, but its smaller and more recent). French Huguenots were another, smaller, population present in the Hudson Valley.
If by “diversity” you’re looking for non-white populations, there were some. It’s anecdotal, but my grandfather, a shoemakers assistant circa 1910, remembered African American workers on the Erie Canal at the time. Albany had a slave market in colonial days, and so there's a long history of slavery in the region, but never in great numbers. African Americans come to the region in part through the Underground Railroad, while not in great numbers they're historically important; so for a fiction writer you have an interesting duality between people of African descent who'd been brought to the region as slaves, and those who'd come for freedom.
After US independence, New York [slowly] emancipated its slaves, with the last slave freed in 1827 and a number of communities of freedmen, often sponsored by Quakers, emerged in the Hudson region. Race relations weren't notably different than elsewhere, beyond the particularities of abolitionists and freedman communities with a special bond. One can find some evidence about concern with "race riots" in the early 1920s in Albany (likely a reflection of the Tulsa story which was in the news); a local Albany insurance broker selling "riot coverage". While mooted, such riots didn't actually take place . . .
see for example:
THE PATROON SYSTEM AND THE COLONY OF RENSSELAERSWYCK
Settlement Variation and Cultural Adaptation in the Immigration History of Colonial New York
Set in Stone: Creating and Commemorating a Hudson Valley Culture
Waves of immigration shape region over 400 years
African American History: A Past Rooted in the Hudson Valley
Have you looked at the 1900 Federal Population Census data? It identifies the nation or state of birth of those enumerated and includes whether or not they could speak English. Where groups of people originating from other nations and their families were clustered can give you ideas about ethnic neighborhoods and what languages other than English were spoken in some instances.