Here are some suggestions in a small list, which is by no means either authoritative or exhaustive -

Books by Herbert Marcuse you should read:

Eros and Civilization

Reason and Revolution

The first, I think, will give you a better insight into Marcuse's thought and it seems to me the more practical book to read. Reason and Revolution is interesting if you want to see how the Frankfurt school took up Hegel.

Adorno's Negative Dialectics is a good book to read to understand the Frankfurt school's approach to philosophy in general and epistemology more specifically. Adorno, once more reading Hegel, shows that while dialectic is critical to philosophy, Hegel was incorrect to think that in the dialectic, the elements are sublated into something greater. Hence, "negative" dialectic. His Minima Moralia is an interesting book and discusses all sorts of topics (like toys...). At any rate, it's one of those books you ought to read if you're going to read Adorno.

Max Horkheimer's Eclipse of Reason addresses how the Nazis were able to make their position seem reasonable. This book is a good engagement with critical theory in a particular historical context. It's interesting to then apply it to one's own context.