Why couldn’t they pacify the land they controlled? Why was guerilla warfare a constant issue?
This is a huge question, if you're a student and have access then check out https://www.jstor.org/stable/23414548?seq=1#metadata_info_tab_contents for a good rundown on differing interpretations.
This question goes to the heart of the debates around COIN (Counter-Insurgency) in the modern Afghanistan/Iraq context. If you haven't I'd suggest doing some reading on COIN as it's arguably an attempt to learn the lessons of Vietnam and apply them to modern conflicts.
From a military perspective tackling insurgencies and guerilla warfare is extremely difficult, because military action that causes civilian casualties and damage to the state often leads to reinforcing the insurgents narrative amongst the population, making them more popular. So by fighting insurgencies, you end up making them stronger. This wasn't helped by U.S. military policy utterly failing to understand this at times. There was a policy where units were expected to provide numbers of how many insurgents they killed. So soldiers would often shoot first and ask questions later to get their numbers up, or would invent 'kills' to make themselves look good to their superiors. So U.S. command was often looking at drastically inflated figures of how well they were doing and making decisions from poor information. Also focusing on 'kills' as the measure of success obscures other ways the U.S. strategy was underperforming in the conflict. (If you haven't heard of the kill count policy check this out https://www.jstor.org/stable/205692?seq=3#metadata_info_tab_contents )
From a domestic perspective, the Vietnam war struggled with support and was hugely divisive, particularly in the later years. Vietnam was one of the first (if not the first) conflicts where images from war could be beamed directly into American households via the media. So the people had to deal with reports of civilian casualties, village burnings and other atrocities as well as the deaths of U.S. servicemembers. This was exacerbated when the U.S. gov lied to its people about how well it was going, partly because of the 'killcount' oriented strategy. So when the U.S. gov has been telling its people that the war was on the cusp of victory and the North Vietnamese orchestrated the Tet Offensive (a massive offensive that involved pitched battles) then suddenly the narrative that the U.S. was on the cusp of victory was broken.
The South Vietnamese state-building strategy was at times dreadfully inefficient. Money was wasted etc but this also fed rampant corruption and South Vietnamese officials got wealthy whilst people continued to suffer the effects of poverty and war, so again the narrative of the insurgency is given more legitimacy in the population's eyes.
Internationally. The flow of Chinese (and Soviet but to a lesser extent) money/weapons allowed the Vietcong and the North Vietnamese army to continue to resupply and remain active. Therefore strategies around attrition, to degrade the insurgency to the point where it could no longer fight, were doomed to fail.
I've tried to focus on U.S. policy answers here because that's the part I've researched in the past and I think that's what you're asking for but there are also other perspectives that focus on dynamics within Vietnam itself that I haven't researched enough to provide here.