Your premise is backward. It should be, "Why didn't soccer overcome baseball" and the answer is "Because baseball was already entrenched as America's Pastime".
I have written about a similar subject to this before in this sub, about why cricket didn't take off in the United States. The answer: it did, but was displaced by baseball before it had too much of a hold. This is due to baseball requiring smaller fields, which was helpful in spreading it in urban areas with limited adequate venues. American nationalism (preferring a home-grown game to the foreign import) and pace of play were also suspected factors. But perhaps most importantly, early cricket clubs discriminated against non-British born players, so American-born players, as well as immigrants from Ireland and Germany, gravitated to the less discriminating baseball clubs at the time that urbanites were joining these types of sports clubs. These clubs formed the first amateur and professional leagues.
The modern game of baseball can trace its rules back the adoption of the 1845 rule-set by the Knickerbocker Club in New York City, which became a standard rule-set which New York-area clubs would play by during inter-club challenges in the 1840s and 50s. The rules may even go back further - one of the original Knickerbocker Club members gave an interview in the 1880s stating that the rules were being used by the Gotham Club he had belonged to prior to that, a club that dated back to 1837.
Regardless, in the 1840s and 50s, the game of baseball became a popular spectator sport in the New York City area, selling lots of tickets and also being a source of gambling (despite the nominal ban on it by the clubs and venue-owners).
There were competing rule-sets outside of New York City, but according to Baseball in Blue and Gray: The National Pastime During the Civil War by George B. Kirsch, the bat-and-ball games generally known as "base" or "baseball" were already being described as a national pastime by the middle of the 1850s:
"In 1856 a Boston enthusiast described that sport as 'truly national,' a game that 'is played by the school boys in every country village in New England, as well as in the parks of many of our New England cities.' He continued: 'Base used to be a favorite game with the students of the English High and Latin Schools of Boston, a few years ago...Base is also a favorite game upon the green in front of village school-houses in the country throughout New England; and in this city, on Fast Day... Boston Common is covered with amateur parties of men and boys playing Base.; Boston’s truckmen attracted large crowds of spectators, who admired their 'supply of muscle that renders them able to outdo all competitors in striking and throwing.'"
According to Baseball’s First Inning: A History of the National Pastime Through the Civil War by William J. Ryczek, the same sentiment was being expressed in New York City at the same time. Minutes of the meetings of the Knickerbocker Club in 1855 described baseball as "the National Game". The sports newspaper the New York Clipper called it the "American Game" the same year and "the National Game" starting in 1856.
As Kirsch points out, by 1860, the game was well-known enough that it could go unnamed as "the National Game" in a political cartoon published by famed lithographers Currier & Ives, using baseball as a metaphor for the contest between the four candidates in that year's presidential election.
Baseball became an organized sport with a national governing body in 1858, under the name of the National Association of Base Ball Players (NABBP), at first made up entirely of New York City-area baseball clubs. It grew slowly in its first years, but member clubs were soon joining from outside the metro New York area. This was certainly aided by the NABBP's publication of their formal rules for the game starting in 1860 in the cheap, nationally-distributed book called Beadle's Dime Base-Ball Player, so-called because the book literally sold for a dime. Beadle's would be published annually for years to come, updating the rules as the NABBP made rules changes to the game.
Even before the the Civil War, according to Kirsch, the game had begun to explode nationally. While the Northeastern cities of New York, Boston, and Philadelphia still remained its centers of dominance, the rest of urban America (especially in the North) had followed suit, and it had spread to rural communities as well. There were 130 active baseball clubs across New Jersey in the latter half of the 1850s, for example, most playing under the NABBP rule-set if not formal members of the organization. Further West, according to Kirsch, prior to the Civil War:
"Larger midwest cities such as Chicago, Cleveland, Detroit, Milwaukee, and St. Paul and smaller towns such as Nininger City, Minnesota territory, Oberlin, Ohio, Davenport, Iowa, Fox Lake, Wisconsin, all reported clubs and matches to the New York sporting journals. On the Pacific Coast, California claimed at least five teams in San Francisco, Stockton, and Sacramento, and several tournaments played according to the NABBP regulations in the fall of 1860 created much interest in the new team sport in the Golden State. A baseball fanatic from Boston, Brooklyn, or Philadelphia had a pretty good chance of finding a ball game as he traveled westward before 1861."
By the end of the Civil War, the NABBP was crowning an annual national championship club. The championship club (the Brooklyn Atlantics) even visited the White House for the first time in 1865. The first openly professional baseball club was started in 1869, in Cincinnati. The NABBP went semi-pro by the end of that same year, with the most dominant clubs going all-pro in 1871, with the breakaway formation of the National Association of Professional Base Ball Players (called the "National Association" for short). While the National Association only lasted until 1875, the surviving clubs re-formed the following season into the National League, which remains as one-half of Major League Baseball to the present day.
By then, baseball had started to be regularly referred to as an American pastime, and since 1864 (when a fair ball caught on the first bounce was no longer an "out"), the game has been played largely in a recognizable form to how it's still played today. The number of strikes for an out, and balls for a walk, were still toyed with for another couple decades, and over-hand pitching was still to come, but otherwise, most of the in-game action has remained the same: nine innings with three outs per team each half-inning, four bases set 90 feet apart, with modern rules about scoring, fly-outs, force-outs, tag-outs, and foul balls.
Soccer had yet to really be standardized in its modern recognizable form at the time all this was happening. "Football" did exist, and had existed for a long time, but just like bat-and-ball games before the 1840s and 50s which could be referred to as "baseball" despite being drastically different from each other, so, too, did "football" refer more generically to a collection of colloquial games without an accepted, widely-played rule-set. And the games that were played didn't really resemble modern Association Football, a.k.a. "soccer".
The older games of football were closer to "mob football" or "medieval football", which were much more "smother the other" style games. They weren't always even played as team sports, just as an "every player for themselves" type of game. They were rather violent, and during the mid-19th century, a lot of schools in America actually banned them because students were getting hurt. At least one school, Dartmouth College in New Hampshire, continued to play the sport uninterrupted, publishing the school rules in 1871 as "Old Division football".
It was out of these kinds of collections of games that not only did the Football Association's soccer evolve, but so did Rugby Union, Rugby League, Australian rules football, American football, and Canadian football. All these games were really only standardized in forms that resemble their modern play-style decades after standardized and organized baseball had been established and was being played as a popular spectator (and gambling) sport in the urban U.S. Northeast in the 1840s and 50s.
The Rugby School in England actually did have a written rule-set by 1845, however, just like baseball's written rule-set that year, it went through considerable evolution. But unlike baseball, where one game took over relatively quickly (the NABBP having really standardized the game by 1858), there was much more conflict in the evolution of the English football games. The Sheffield Rules were established in 1858, and football clubs attempted to form a national Football Association in 1863, but there was disagreement of which rule-set to use, and the Rugby School ended up not joining. They started their own association in 1871, which became the Rugby Union. The Football Association (sometimes called the "London Association" in its early years) adapted the Sheffield Rules into a new rule-set called the "London Rules", but even those weren't entirely like the modern game of soccer. Notably, a player could catch the ball in the air, and received a free kick from where they stood - and the goal did not have a top cross-bar, so goals were scored between goal-posts much more similar to modern rugby, Australian football, and American football.
(cont'd...)