The only way to explain Hoover's neglect in dealing with the Italian-American Mafia prior to the Apalachin Meeting in 1957 is to engage in speculative history. As Hoover was both incredibly powerful and intensely secretive, there is absolutely no way to know for sure what his reasoning was for avoiding serious investigation and prosecution of the Mafia before Apalachin. There are, however, at least three well-accepted theories about this, all of which are reasonably believable.
First, Hoover likely avoided going after the Mafia because the Mafia had found evidence of Hoover's homosexuality. According to biographer Anthony Summers, whose book Official and Confidential: The Secret Life of J. Edgar Hoover detailed the allegations about Hoover's sexual life, the Mafia obtained this information two ways. The first was that there were photos of Hoover cross-dressing circulating around secretive channels that the Mafia likely obtained. These photos were probably taken by the Office of Strategic Services (OSS) in the aftermath of WWII as Hoover and OSS Chief William Donovan were feuding over who would control American intelligence service. The second way Summers suggests is that Hoover was caught engaging in homosexual behavior at a sex party by the wife of liquor distributor and Mafia-ally Lewis Rosenstiel. While Summers' claims are difficult to verify, the fact remains that rumors about Hoover's homosexuality had existed as early as the 1940s and if the Mafia had proof they would have certainly used that proof to blackmail Hoover into turning a blind eye to their activities.
It is also probable that Hoover avoided prosecution of the Mafia because the FBI relied on Mafia figures as informants. Because the Mafia had information on the who's who of the criminal underworld they could provide information about Hoover's actual targets, whether Communist spies, Civil Rights leaders, or public figures or politicians suspected of engaging with the USSR. While secret meetings between Hoover and Mafia leaders would most definitely not leave a paper trail for the historical record, the Justice Department basically confirmed collaboration between the FBI and the Mafia in 1997 when they supported small-time criminal Joseph Salvati's lawsuit that he was wrongfully convicted of murder due to the FBI and Mafia leaders acting in cahoots to protect higher-up informants.
Finally, while there are too many Hoover biographies to list here, there is consensus among historians that he had particular goals which changed at various points during his career and organized crime took a back seat between about 1935 and 1957 for the simple reason that Hoover's personal goals as a lawman changed. This is largely because Hoover spent much of his early career as a gangster hunter. He caught noted bank robbers such as John Dillinger and Machine Gun Kelly during this time and by 1935 when the FBI was created, Hoover put his gangster days behind him and used his newly increased investigative resources and manpower to pursue what he saw as a the "big fish" of the criminal world. Gangster's were petty when compared to the Nazi and Communist collaborators that Hoover sought after 1935.
My personal take on the matter of Hoover and the Mafia is that all three theories are somewhat valid, however in order to understand Hoover I find it is best to take a broad look at the time and place he worked in as the Director of the FBI from 1935 to 1972. During these years Washington DC was run by a generation of men with particular values and imperatives about national interest. They were incredibly competitive, masculine, secretive, and aggressive. As such, they placed the highest value on heroism against enemies domestic and foreign who threatened their American ideal. The Mafia was not one of these enemies. Rather, the values of the Mafia meshed quite nicely with the values of government men in power. As a result, they were not aggressively targeted. In short, while Hoover was certainly a powerful individual, he was only a single part of a generation whose ideas and values determined history moreso than any one man.
J. Edgar Hoover biographies are too many to list here and, in my experience, reading them only drums up more speculative questions and tabloid-type fodder that has little value in meaningful historical speculation. My book recommendation for those interested in understanding Hoover and the goals of his generation is Imperial Brotherhood: Gender and the Making of Cold War Foreign Policy, by Robert Dean