This is a fascinating question not only for the big picture "answer" but also as a way to examine the evolution of the historiography of the question itself as it relates to Gettysburg, Lee's objectives, and the broader question of where the conflict stood in June/July 1863. While some elements of what u/slcook said are correct, much of it is based on a now-outdated understanding of not just Gettysburg as a battle, but the conflict as a whole. This includes a re-examination of Lee's Pennsylvania objectives, Stuart's role in the battle, and the thinking about Vicksburg in June 1863.

Part of this is due to the sources u/slcook references at the bottom of the post, which are curious. Ken Burns' 'Civil War' documentary, while a very engaging series, is not an ideal source for serious modern scholarship. This sub's FAQ section has a great write-up that gives more detail (https://www.reddit.com/r/AskHistorians/comments/54zv3w/ken_burns_civil_war_series_i_love_it_and_have/d86qc8w/) but in sum, it shouldn't be used as a guide for a modern understanding of the conflict and its particulars. The same goes with Shelby Foote, which like Burns' series, is a great guide (and something that can sometimes be cited with other sources...carefully) but not something a modern historian would look to as a primary citation source for good, updated scholarship. The Sun Tzu citation is its own thing, and while I understand why it was included, doesn't really pertain to the actual history of the engagement.

McPherson is a great source, however, which makes it all the more curious as an addition to the comment by u/slcrook. McPherson makes no mention of Lee's desire to win a victory to force the north to sue for peace (even though the 1993 film Gettysburg weirdly makes this claim as an absolute, which is perhaps why it has bled into the popular consciousness). McPherson DOES spend quite a bit of time talking about how Lee's invasion of Pennsylvania was mostly a knee-jerk reaction to Jefferson Davis' suggestion that he head west to help out with Vicksburg (pg. 646). Long story short, Lee had to do something if he was going to stay out of Mississippi, so he went north to Pennsylvania looking for supplies, fresh recruits, and a piecemeal victory over what he knew to be a larger foe (the Army of the Potomac).

This is important to understand, and something historian Stephen Sears also touches on in his book 'Gettysburg' (2003). Lee wanted to ultimately destroy the Army of the Potomac, sure, but he hoped to do this piecemeal by catching it in fragments, strung out. He hoped this victory and the invasion would bring him (1) foraged supplies, (2) new recruits, and (3) a public victory that would further weaken the Union's willingness to continue the war. Capturing Washington City (it was not known as "D.C." at the time) and/or forcing the Lincoln administration to sue for peace was never seriously expected.

So to the broader OP question of why Lee persisted on the 2nd day at Gettysburg, we should dismiss this notion that he HAD to in order to secure his ultimate campaign objective. Lee didn't go to Pennsylvania looking to score a set-piece victory between fully engaged armies: he wanted to feed off the largely untouched Pennsylvania countryside for supplies and new recruits (both in short supply in Virginia) and to catch the Army of the Potomac in fragments.

Stuart's participation is a little trickier, and is something we need to take a harder look at in the 21st century. Much of the narrative around Stuart's "galivanting" in the days leading up to Gettysburg has been a part of a broader Lost Cause myth to excuse Lee (and by extension, the Confederacy) from blame in its ultimate defeat. In short, in Civil War myth-making, there are heroes and villains. For Southern myth-makers Lee was the hero and everyone else was expendable. This meant that Lee wasn't REALLY to blame when he went against Longstreet's advice not to attack on July 2nd, it was REALLY Longstreet dragging a$$ and sabotaging the "great" Lee's plans (hopefully the sarcasm in that statement is obvious). Likewise, based on this understanding, it wasn't Lee's fault that he foolishly committed his smaller army against a numerically superior, better positioned Union foe, it was Stuart's fault for not being there to warn him.

Modern scholarship takes a more nuanced view of this. Lee was not especially precise with his orders to Stuart ahead of the Pennsylvania invasion, and had commended the cavalry commander in the past for taking the same initiative that he did ahead of Gettysburg. Sears spends a great deal of time on this in his book, but to summarize, Lee gave Stuart very broad orders, none of which bound him strictly to reconnaissance work, and Stuart's reading of these orders can't really be faulted for Lee's disposition on July 1st. Stuart was never explicitly ordered to stay nearby to provide close screening or scouting support, and if he hadn't run into delays getting back to Lee's main body, nothing at all about his movements would have been out of the norm. He did run into unexpected trouble on his ride around the enemy in late-June, and having Stuart with Lee's army would have changed the course of the battle without question, yet his actions leading up to Gettysburg reflect a freedom of movement and operation that had been afforded to him in the past (which he had indeed earned and always been commended for). Again, though, it's complicated. Stuart would have been wise to stick closer to Lee, but he had no way of knowing how delayed he'd be in getting back, and such action in the past had worked out well for his command.

Now, with all of that out of the way, to get back to OP's question of why Lee stayed after that first day: he thought it was the right thing to do, from both a strategic and morale perspective. Yes the Union held the high ground in spots, but they were not yet fully dug in, and in some areas (like the Round Tops) they appeared ripe for assault. As McPherson writes in 'Battle Cry of Freedom' (pg. 655-656) while Longstreet, "concluded that this line was too strong for an attack to succeed" and urged redeployment, Lee felt, "The morale of his veteran troops had never been higher; they would regard such a maneuver as Longstreet suggested as a retreat, Lee thought, and lose their fighting edge." It wasn't that Lee HAD to fight lest this mythical ebb of the war's momentum roll over him, it's that he felt that his army was in as good a shape to win as they ever would be. The men under his command were hungry for a fight, and he recognized this as the same spirit that had carried them to victory at Chancellorsville, Fredericksburg, and 2nd Bull Run. It was a gamble, to be sure, but one that fit into Lee's larger strategy of engaging the Army of the Potomac when it was still strung out and not fully arrayed for an engagement (which it wasn't when planning began for the second day in the late hours of July 1st).

This myth that Lee went north looking to land a killing strike to end the war is fanciful, and a distortion of a far more complicated truth. When Lee began planning the campaign in June 1863, Vicksburg's fall wasn't a foregone conclusion (though it was in a tight spot and needed help), and the thinking was that if Lee could cause enough trouble in the east, it might force Lincoln to move resources from the west, thereby taking some of the pressure off of Mississippi.

TD;LR - Lee attacked on the 2nd day at Gettysburg not out of desperation, or because his ultimate objective was to win a grand battle to end the war, but because he genuinely thought it was the best course of action in the moment.

Edit to add that the comment linked above from the FAQ section regarding Ken Burns' Civil War documentary series was provided by u/Georgy_K_Zhukov