Put simply, it was not an objective of the campaign, and actually would have hindered its completion. The main body of Sherman's right wing never came closer to Andersonville than Griswoldville, with Kilpatrick's screening cavalry never getting much closer than Macon (roughly 60 miles away from Andersonville). I wrote previously about Sherman's objectives during the campaign, here: https://www.reddit.com/r/AskHistorians/comments/34zcc4/was_shermans_march_justified_why_or_why_not/cqzm1lf/ - but in short, Sherman's objective was to take Savannah, an important Confederate seaport/city/stronghold, and simultaneously disrupt the South's ability to make war by smashing manufacturing and transportation infrastructure on the way. The added bonus of inspiring Confederate desertions as a result of the destruction wrought along the way was a plus as well.

This campaign was put in motion in concert with Grant's actions against Lee out east, and was meant to put pressure on the Confederacy from both directions (west and east). The reasoning behind this was that with any luck, Sherman could carve his way to the coast while Grant defeated Lee (or bottled him up as it turned out). Although thousands of Union prisoners suffering in unspeakably foul conditions was a terrible tragedy, Lincoln, Grant, and Sherman all recognized the greater tragedy: the continuation of the war. With this in mind, all three men (and many more besides) resolved to end the war as quickly as possible, relieving the suffering of EVERYONE with a final, complete Union victory

Sherman pausing the campaign to liberate a prison camp would have (1) forced him to halt a crucial, time-sensitive campaign to deal with tens of thousands that needed medical care, and (2) would have likely made many of those soldiers he did have in fighting shape sick from the exposure, and (3) exacerbated an existing problem his army had with African-American refugees following both wings of his army.

Sherman was on a different humanitarian mission, one that offered a far better life-saving opportunity than liberating one prison camp. It sounds harsh, and likely wouldn't have made a prisoner at the time happy to hear, but by late-1864, the "arithmetic" as Lincoln put it, was clear. To save lives, to be humane, they needed to end the war, and that's what Sherman helped to do by continuing his march to Savannah.

[Sources: James McPherson, 'Battle Cry of Freedom'; Doris Kearns Goodwin, 'Team of Rivals'; Stephen Sears, 'Lincoln's Lieutenants']