H.G. Welles's The Time Machine (1896) is commonly credited with popularizing the idea of time machines. However, it only deals with traveling into the future, never even bringing up the idea of going back to the past. What story popularized the idea of going to the past and/or changing the present?

There are 2 types of time travel, via a time machine or other means.

Samuel Madden's Memoirs of the Twentieth Century (1733) might be considered the first time travel story in which a guardian angel takes some letters from 1997-1998 back to the present (present 1733 at the time).

While it's more of a go to the future and back to the present, it has the concept of going back in time and tries to change the present by letting people know how the future will look like.

Next in line would be Charles Dickens' A Christmas Carol (1843), this one depicts time travel in both past and future without any time machines.

One of the first time travel stories involving a time machine is "The Clock that Went Backward" by Edward Page Mitchell (1881). But this one involves an actual clock and it can be disputed if this can be considered a time machine or if it's just fantasy.

Enrique Gaspar y Rimbau's El Anacronópete (1887) can be considered one of the first time travel stories with a time machine specifically engineered for time travel and it also depicts time traveling in the past.

I don't know which one/s actually popularized the idea of traveling to the past or changing the present. My take would be on Edward Everett Hale's "Hands Off" (1881), this tells the story of an unnamed being, possibly the soul of a person who has recently died, who interferes with ancient Egyptian history by preventing Joseph's enslavement.

He was considered a pioneer in time travel and stories that change the past (which automatically would also change the present).