The answer to your question lies across several decades, stretching from the 1880s to the present day.
In the early days of Italian immigration the Mafia was little more than a ragtag group of ne're-do-wellers that preyed on their own kind in the ghettos and slums of New York, Boston, Chicago, St. Louis, etc. This criminal element was known as the "Black Hand" and really posed no threat to the society at large. Their victims were fellow Italians struggling to make a living. The romanticized version of the mafia is just that - romanticized. They were predators that got a foot hold by strangling their own community.
The Black Hand specialized in extortion, gambling and loan sharking within the Italian community. It terrorized its own people and shook them down at regular intervals. While it is cliche today, it was very common for the Black Hand to force owners to pay "protection money." The guise was that the Black Hand would protect them from crime. However, the crimes were perpetrated by the Black Hand. A common crime was breaking store windows. A more severe crime was firebombing.
There were multiple Black Hand groups operating in the Italian communities. At times, these groups were not on the same page and would attack each other over "turf." At other times, they would clash with older, more established criminal groups - in particular the Jews and the Irish. In fact, the preeminent gangsters of the early 1900s were Bugsy Segal (Jewish), Meyer Lansky (Jewish) Bugsy Moran (Irish) and Machine Gun Kelly (Irish).
Over time, the Italians became the leading force in underworld crime. Their dominance was achieved through violence. In the 1930s and 40s, the Italians realized that they were wasting time, energy and money by killing each other. The solution was to formally "organize" and set up a structure that provided for maximum profit and peace while diminishing conflict and government investigations.
This structure allowed the organization to grow in leaps and bounds. It's "reasonable" approach to crime placed it a cut above other hoodlums and created an atmosphere of tolerance within government and policing. From here, corruption ran rampant. There were always tumors that the Italian mob had police officers in its pocket - a fact later confirmed by the arrests of two NYPD.
With these connections in place, the mob was allowed to run rampant. It quickly involved itself in all major unions and labor organizations. It got to a point where virtually nothing could be shipped into or out of New York without the Mafia being involved. It controlled the docks and the teamsters. It was the equivalent of Carnegie owning the coal mines and the railroad connecting to the coal mine. Speaking of shipping - New York was a megahub for global trade. Need something illegal to pass inspection at the docks? Need to sneak in a metric ton of cocaine? You need to speak with someone in the Mafia. If they got paid, you got a pass.
At this point, the Mafia could now impose its will on the politicians. No politician wanted to admit there was a mob problem - so it placated the mob at every turn. The mob could threaten to strike across several unions, crippling the city. Every politician wanted to avoid that at all costs. There are also instances of politicians being in power because of mob influence on elections. Favors like that need to be repaid...early and often.
So, if you control the police, a few politicians and a few important judges - you have insulated yourself from prosecution. That allows you to grow, uninterrupted, while your enemies are constantly beset by legal problems. You control organized labor throughout the city. Nothing gets built without mob involvement. You control every racket (gambling, loan-sharking, prostitution, drugs, illegal firearms). You set up legitimate companies to bid-rig major contracts with the city. There was virtually nothing that happened in New York City without the mafia getting paid.
At this point, you may ask why no one unseated the Italian mafia. The simple reason is that, as brutal as it may be, the Italian mafia always valued business ahead of violence. It was much more economical for the Italians to "allow" Albanians, Russians, Chinese, etc. to operate and kick back than it was for the Italians to openly kill other groups. A kick back doesn't make the front page of the Times. Gangland murders usually do. So, the Italians were enterprising and their "reasonable" approach to crime allowed them to operate somewhat peacefully with many other groups.
In answer to your question - the Italian Mafia ruled over everything in New York for decades.
But what about now? Currently, our politicians would have you believe that the Mafia is a shadow of its former self and that it's hey-day is long over. Certainly, the federal investigations doomed its unfettered growth. Mega arrests happen often and two of the five families have a boss serving hard time. The families have banned killings - unless it is an absolute necessity. Killings draw too much attention and can lead to sweeping charges under a RICO case.
In the mid 2000s, the FBI essentially declared the mob dead. It boasted that the mafia was out of the labor unions and off the docks. It owned no construction companies and was essentially relegated to minor player in minor street crimes. Then in 2011, the FBI made hundreds of arrests of mafia members that had involvement with the unions, the docks, and construction companies. That's plenty of activity for something dead.
The mafia's current leadership avoids the limelight. It is run in secret - the principle upon which it was founded. It not being on the front page is by design and not demise.
Some interesting sources: