Religion isn't necessarily banned in China, they just don't allow it to make decisions for the state. Chinese citizens are allowed to follow 5 religions - Buddhism, Taoism, Islam, Catholicism, and Protestantism. The Chinese constitution specifies...

Citizens of the People's Republic of China enjoy freedom of religious belief. No state organ, public organization or individual may compel citizens to believe in, or not to believe in, any religion; nor may they discriminate against citizens who believe in, or do not believe in, any religion. The state protects normal religious activities. No one may make use of religion to engage in activities that disrupt public order, impair the health of citizens or interfere with the educational system of the state. Religious bodies and religious affairs are not subject to any foreign domination.

So basically, religion is allowed, but using religion to influence the state isn't.

Though there have been many communist countries, the relationship of religion and communism has varied a great deal between them, depending upon which parts of Marxism they embraced on a philosophical level.

Marx famously said that 'religion is the opiate of the masses'. Now this needs to be placed within the Marxist idea of structure and super-structure, that is that the culture and identities of groups within society (the super-structure) are determined by the economic structural foundation of that society. In the capitalist and pre-capitalist phases, Marx saw the role of religion as a way for the masters to escape their guilt, and the workers to escape the harsh reality of exploitation through the possibility of gaining reward in a (non-existent) afterlife, and that once socialism had been achieved, religion would die a natural death as people, living within the socialist structure and dialectic materialist super-structure, would no longer have any need for the supposed benefits of religion.

Lenin's ideas about religion developed from Marx's starting point, that religion was not in the interest of the workers, but also saw that the hierarchical nature of the church acted to hold back the active development of socialism, and so, rather than Marx's malign disinterest, approached religion as a force to be actively combated. This took the form within the early Soviet Union of an acceptance within the constitution of the right of the people to worship and of freedom of belief (or non-belief), but that the Party, as the vanguard of the people, should work actively to persuade the populous that atheism was the correct position. Also where the church had supported the Whites (which was most places in the Soviet Union) the organised nature of religion should be suppressed and the corruption that was the church's possession of land and wealth confiscated.

This position was maintained by Stalin and pushed forward even faster and more forcefully as with many aspects of Stalin's crash programmes towards achieving socialism, with most of the remaining priests and religious communities being forced from their churches into the camp system or prison colonies in Kazakhstan and Siberia. This position was softened after the first year of the Second World War as an expedient to unite and motivate the people to fight and improve foreign relations, but by the time of Stalin's death in 1953 was slowing being reversed once more.

The degree to which each communist state followed the orthodox Moscow line varied over time, with the 1950's being the highpoint of religious persecution in Eastern Europe, but following the 1956 secret speech by Khrushchev, and the unrest that followed in Poland and the Revolution in Hungary later in the year, the communist nations began to follow different courses on religion.

Hungary, being largely Catholic, initially resisted communist control of the Church, but with the confinement of Cardinal József Mindszenty, nominal leader of the Hungarian Catholics in the US embassy from 1956 through until 1971, eventually largely acceded to Communist demands over influence over church appointments and limits on worship, though this was less demanding due to the more relaxed policies of Kadarism than other Warsaw pact nations.

Czechoslovakia had, before the communist takeover in 1947, a long history of religious strife between the Protestant and Catholic communities with the later dominant in Slovakia, and the former dominant in the Czechlands. The communists capitalised on this religious difference to very effectively destroy the official church structures and render anything other than small home-worship groups more or less impossible outside of the few hundred strictly state controlled churches, which only liberalised very slightly in the late 80s just before the collapse of communism.

Poland, with Catholicism at the centre of cultural life and Polish identity managed to resist communist control quite effectively through much of the 60s and 70s, had representation in Parliament, and maintained a role in education throughout the communist period, even having its own University only partially subject to communist control. In the 1980s the Church would play a very vocal role in resisting Communism and the martial law that was imposed for much of the decade.

Mao was not happy with the Soviet bloc's de-Stalinisation and kept closer to the Stalinist line with regards to religion, as did the Maoist Albanian state, where there was no particular strong religious heritage among the population historically, having been a battleground between the Christians and Muslims for centuries, and became the only state to fully outlaw religion entirely. North Korea also, as it still does to a great extent today, maintained an ultra-Stalinist line, and so was extremely anti-religious.

This (long) post doesn't answer the question directly, as to Cuba's Catholicism (I've not formally studied Cuba), but I hope it goes a long way to show that from the initial views of Marx, both changing intellectual positions and longer term historical factors meant that each Communist nation ended up addressing the question of religion in a unique way.

As for sources, the various books by Sabrina P. Ramet are a great place to start, but much of this is based upon my own primary research.

Edit: proof reading

Cuban Catholicism and clergy are heavily influenced by liberation theology and do not represent a counter-revolutionary class that could challenge the Cuban state. As a result, there is no benefit for the Cuban state to attack Catholicism, but a significant risk of turning the population against it.

This was not the case with the Orthodox Church in Russia, so the Bolsheviks were much more aggressive in stamping it out, though a neutered form did survive in some areas.

As far as I know, Albania is the only country to have "outright banned religion." Although now legal, most of the country(60-70%) are still nonbelievers.

Neither China, North Korea nor the USSR ever "outright banned religion," although they certainly did everything they could to discourage and replace it. As some posters have pointed out, the USSR first restricted the Orthodox Church, then co-opted it during the war. In China and North Korea, religious affairs are conducted through (heavily state-controlled, and probably perfunctory in the latter case) patriotic religious organizations. Even during the Cultural Revolution it was legal to do religious things(but also really, really dumb)

To answer the OP's question, the Communist Party in Cuba seems to have learned a lesson from Poland. Like Cuba, Poland tried to discourage Catholicism, with disastrous results for the regime when the workers sided with the church.

On mobile, sauce when I get to work.

Cuba is not an atheist state, and has a population of mostly Catholics.

Christians in general have more organized religion, which means more powerful churches. It is difficult to remove Catholicism from a country (not including purges).

Secondly, there is no real reason to do so. While the USSR may have encouraged atheism in it's political thinking, the idea of socialism has mutated and changed in the countries it has spread to.

Didn't the USSR allow at least some freedom to the Russian Orthodox Church?