In the sociological literature, this is the question of "secularization" and doesn't just cover atheism, but religious affiliation more broadly (out-and-out atheists are often a minority of the religiously unaffiliated). I go through all the main causal ideas for secularization here. In Eastern Europe, religion's closeness to the (unpopular) Communist regime and importance religion for national identity. This is the basic post, explaining how secularization works, but I don't specifically apply its findings to Eastern Europe.

I go into that a bit in this shorter post comparing the Czech Republic and Poland and Slovakia. To be a good Czech (proud of its Hussite heritage) or good German (East Germany was historically mostly Lutheran, but Germany in general was religious divided) didn't involve being one particularly kind of Christianity. On the other hand, to be Polish and Slovak implicitly means to be Catholic, to be Russian meant to be Orthodox. Ukraine is an interesting example: conflict between the self-declared political representatives of different identity groups seems to have reinforced ideas that everyone has to be "something" even when individuals are not necessarily pious (a similar thing can be seen in the Bosnian War; someone quipped, "Muslims, Catholics, and Orthodox are all killing each other over religions none of them believe in").

As I go through in this post, Catholicism generally was better at resisting communist influence because they were an international movement whose leaders ultimately reported to Rome, whereas the top local Protestant and Orthodox leaders could be pressured and replaced more easily by Communist apparatchiks. Particularly scroll down and see the contrast in the Baltic states, where Catholic Lithuania and formerly mostly Lutheran Estonia and Latvia provide a good "natural experiment". They all received similar treatment under Communism, but Lithuania is mostly religiously affiliated and Estonia and Latvia are mostly unaffiliated. Because of their entirely local organizational structure, Protestants were particularly susceptible to leadership cooption (though this isn't always the case--Jason Wittenberg in this Crucibles of Political Loyalty goes through how the Protestant minority in Hungary, mostly Calvinists, was able to have strong continuity and become a center of anti-Communist agitation). But looking at Latvia and Estonia shows that East Germany and the Czech Republic's low rates of religious participation aren't unique in Eastern Europe.

The closeness to the old regimes also matter. In the Czech Republic, I've been told, the Catholic Church was mostly coopted. Resistance to the Old Regime came through atheist intellectuals. In Poland and Slovakia, on the other hand, resistance was organized a lot through the Church (you see famous pictures of priests blessing people in Solidarity). There's also closeness to the new regime; Putin most notably, but Polish political parties and others in Eastern Europe have encouraged religious affiliation in their post-Communist order. Many right-wing strongman countries have particularly encouraged the traditional religious group to flourish. In Germany and the Czech Republic, in modern times, displays of piety are not expected, but in a place like Poland or Russia, such displays could be useful for ambitious men in business or politics.

There is another point I do not seem mentioned there. The religiosity is usually positively correlated with how rural the area is. Bohemia and East Germany were urban territories compared to Poland or Belarus. Even on the scale of regions inside the countries, this correlation can be observed.