Perhaps someone else will write a better response to this, but in the meantime here is something I wrote on the topic a while ago in a MapPorn thread: https://www.reddit.com/r/MapPorn/comments/8jvxch/james_k_polks_expansionist_ideas_for_the_united/dz49sfc/

Since that post was in response to a particular map I'll copy and edit it below. My main source is the book "The Shaping of America, Volume 2: Continental America 1800-1867", by D.W. Meinig.

So, President Polk never wanted to annex all of Mexico. He was most interested in acquiring California and especially San Francisco. Before the war he tried to purchase territory north of New Mexico west to California, including San Francisco, with options and additional money for a more southernly line to include Monterey, CA, and even more money for New Mexico as well. Mexico rejected all this, so Polk instigated a war.

After the initial US victories, Polk tried again to purchase the frontier provinces. Mexico refused and Polk ordered an invasion of the Mexican heartland in order to force capitulation.

After the capture of Mexico City there was a truce and negotiations began in earnest. Of note, it was only a truce, not a surrender, and war could have resumed. In other words, although Mexico had been defeated militarily they had not actually surrendered and were not powerless in the peace negotiations. Still, annexation of some of Mexico was inevitable. Polk and most or all of his administration, and most or all of Congress (and most Americans in general), did not want to annex the densely populated core of Mexico.

In fact, Polk and some of his cabinet wanted to annex only the territory north of 26°—that being the latitude of the mouth of the Rio Grande. There were some in his administration who wanted to annex more, for "defensive purposes". These people, including Sam Houston and Jefferson Davis, proposed annexing territory north of about 22°-23°, around the eastern port city of Tampico, and perhaps not quite as far south on the west coast. Exact proposals differed, but generally they avoided the densely populated areas of central/southern Mexico, drawing lines north of the modern states of Veracruz on the east and Jalisco and Nayarit on the west.

Although these proposals stopped short of the dense core of Mexico they still including a large amount of Mexican territory and a large number of Mexicans. As far as I know Polk never supported these ideas.

On the other side of the coin, there were others in Polk's cabinet who did not want to annex even as much as Polk's 26° line. These people were more focused on the harbors of California and wanted to minimize the number of Mexican people annexed (not to be nice to the Mexicans so much as being worried that Mexicans would not make good citizens, were "unamerican" foreigners, unused to democracy, and would generally result in more trouble than gain for the US—a point of view that today looks quite racist or at least rife with anti-Hispanic prejudice).

These factions argued for a boundary along the Rio Grande and a line along 32° to the Pacific (which would include San Diego and is close to the present boundary). Polk eventually agreed to this Rio Grande and 32° line as the "minimum acceptable", with Baja California desirable but not essential.

Mexico was willing to give up San Francisco early in negotiations, but tried hard to keep New Mexico. And there were some Americans who were quite willing to forego New Mexico. In the Senate there was an attempted amendment to the treaty that would exclude "New Mexico with [its] population [...] of hostile people unwilling to be united to us, and unfit to be trusted with a participation in our free form of government, the principle of which they are utter strangers". The amendment was defeated 18-30. The wording quoted above shows the kind of attitude that made many people want to annex as few Mexicans as possible while still gaining value territory, especially ports on the Pacific.

Once the negotiations reached this point, where the larger annexation proposals of people like Houston and Davis were rejected (by both Mexico and Polk), and the Rio Grande would be the border in the east, the only real place left for compromise was the boundary between the Rio Grande and the Pacific. Mexico tried for the Gila River and a line west to a point just north of San Diego. The negotiator Nicholas Trist at one point agreed to 33°, north of San Diego, but was rebuked by the Polk administration. Mexico was firm about keeping a land connection to Baja California. The US agreed to this but insisted on navigation rights on the Colorado River through Mexican territory—this seems almost funny in hindsight since that part of the Colorado River barely flows at all today.

In the end the boundary agreed to followed the Gila River and a line close to 32°.

So, in short, the primary reason why the US did not annex more of Mexico—and definitely not the dense heartland—was prejudice, worry, and even fear of incorporating so many Mexican people into the US. Similar issues had come up in the aftermath of the Louisiana Purchase, when a great many French and Spanish people had been annexed/purchased into the US. Most Americans believed that the French and Spanish people of Louisiana were incompatible with the US, culturally and politically. Jefferson, and others, came up with various schemes intended to "dilute" the population with "real Americans", and to apply pressures of assimilation. This is part of why the state of Louisiana is much larger than the territory that had been well populated by French and Spanish settlers at the time: To "make room" for American settlers who could be induced or even paid outright to settle in Louisiana and take the lead in forming a properly American system of government.

For various reasons things worked out fairly well in Louisiana fairly quickly. Still, the population of what became the state of Louisiana was somewhere around 75,000 at the time. Compare that to Mexico in 1845, which had about 7,000,000 people, mostly in the central and southern parts. Most of the territory annexed had very small numbers of Mexicans living there. New Mexico had the largest Mexican population, at about 70,000 or so—which is why some US politicians did not want it.

That even 70,000 Mexicans was seen as a problem to the US government can be seen in the repeated rejection of statehood for New Mexico for over 60 years. New Mexico met all the traditional requirements for statehood (eg, a population of at least 60,000), and properly petitioned Congress for statehood many times, starting almost immediately after annexation. It wasn't until 1912 when statehood was granted, which not uncoincidentally, as Meinig points out, was about when the "Anglo" population finally started to equal the Hispanic. By 1912 the population of New Mexico was over 300,000. This process contrasts quite starkly with Nevada's statehood. Nevada went from territory to statehood in just 3 years. When Nevada became a state in 1864 it did not even have the 60,000 people traditionally required.

TL;DR: Basically, racism.