Very little fighting took place in the town itself, but the first day of battle took place almost entirely in a radius of a mile around Gettysburg, primary to its west at McPherson's Ridge. General Henry Heth, leading a division of A.P. Hill's 3d Corps, accidentally encountered and then clashed with John Buford's Federal cavalry along the Chambersburg Pike (road leading into Gettysburg from the northwest). The growing din of battle lured more troops into the battle, eventually drawing a battle line that formed a semi-circle from Hagerstown Road through McPherson's Ridge and Barn to Rock Creek, north of the town. In the afternoon of July 1, the Confederates under Jubal Early attacked and overwhelmed the weak right flank of Buford's cavalry, obliging them to retreat. At about that same moment, Generals Heth and Pender suddenly surged and soon the Federal I Corps was also on the run. The Federal troops fled south through the town, harried all the way by the Confederates, into the hills south of the town. Source: Time-Life's Echoes of Glory, "Illustrated Atlas of the Civil War," pages 114 (map of day 1) and 115 (text).

It wasn't so much as the town was indefensible as the wooded hills like Culp's Hill, Big and Little Round Tops, etc., were very defensible and also not crawling in Confederate troops. Hope that answered your question sufficiently.