It’s not a silly question. It’s an excellent question and one that the toughest brains in the US Air Force and US Navy had been pondering for years, pretty much until 1991, when the cosmic success of allied air forces in the war against Iraq reinvigorated the trust in American tactical aviation. However, even though the numbers themselves are high, the are not disproportionately high in the context of the threats that US aircraft had to face in the Vietnam War (or American War, if you’re Vietnamese). So the question we really need to ask is indeed not “why,” but rather, “how?” And in the process I’ll need to dismantle a few misconceptions.

So, where do we start? Ah, yes, bullets. If you have an AK rifle and happen to be close to a hovering helicopter, sure, you can spray your 30 bullets in the hope of damaging the engine or killing the pilots. But in terms of bullets that’s not where the true danger to aircraft lies. Against aircraft, one would use dedicated anti-aircraft weapons ranging from heavy machine guns rigged with anti-aircraft sights to heavy 100 mm anti-aircraft artillery (AAA for short).

Your assumption that bullets are faster than airplanes is correct (for the most part), but if you want to shoot down a plane with a machine gun, you won’t fire so that your bullets chase after the target closing up on its tail. The fundamental principle of anti-aircraft gunnery is called deflection. This means that you calculate or assess where the target will be after a specific time and shoot at that place, not at where the target is now. It’s easier to show than to describe, so here’s a diagram. As you see, if it takes 4 seconds for the bullets to cover the distance to the target’s current location, the target will have moved away by a distance of 700–800 meters while moving at a speed of 700 kph (the numbers are just an example and don’t represent any specific situation). Thus, you need to predict where the target will be, and aim there, even if the distance that the bullets need to cover is longer.

The next part is that planes can withstand small arms fire. Kind of… Depends where the bullet hits. But as we’ve already mentioned, small arms is not the method of choice for dealing with aircraft, and heavy machine guns form the lower end of the spectrum. At the upper end there were weapons such as the 85 mm 52-P-365 and the 100 mm KS-19. If a nearby explosion of a heavy fragmentation round peppers your plane with shrapnel, it might cause damage to the fuel system, to the engine itself, to the flight control actuators, and to you—the pilot—as well.

What also needs to be pointed out is that the North Vietnamese forces absolutely did have radar equipment. The SON-9 (NATO code name: Fire Can) and SON-50 (NATO name Flap Wheel) radar stations were used to direct anti-aircraft artillery.

Why not just fly higher? The KS-19 can easily reach upwards of 10 thousand meters (32 thousand feet). But also, as you’ve noted, some missions require you to fly low. And that’s not just CAS, but also search and rescue missions and virtually any ground attack mission, since a large part of the Vietnam War had been fought before viable laser-guided bombs were introduced. In general, choosing the altitude from which to drop bombs was a matter of fine balance between accuracy, exposure to AA fire, and risk of damage the attacking aircraft from the bomb itself. In general, all bombs except the Mk 82 Snake Eye were dropped from above 1,000 feet.

On top of that, flying low can help you avoid enemy radars and achieve surprise, yet it also puts you at risk of stumbling across AAA. Just as an example, here’s a summary of an attack on Uong Bi powerplant near Haiphong (from A-6 Intruder Units of the Vietnam War by Rick Morgan):

Only three aircraft were involved – two Intruders and a sole E-2A from VAW-11 Det C for communications. Kitty Hawk commenced launching aircraft in radio silence at 0100 hrs, undoubtedly in deference to the Soviet intelligence ships that normally shadowed the Seventh Fleet. Squadron XO Cdr Ron Hays, with B/N Lt Ted Been and their wingmen, Lt Bud Roemish and Lt Cdr Bill Yarbrough, rendezvoused and flew ‘feet dry’ [over land] at an altitude of 500 ft or less until 25 miles from the target, where they started a slow climb to a safe bomb-release altitude of 1800 ft.

The additional problem is, of course, that at low altitude you don’t have time to try to work around any issues which might present themselves. If your engine goes out, you won’t be able to restart it when descending from 500 feet. This makes you more vulnerable to AAA but also to accidents, including quite simply flying into the ground. Additionally, many aircraft classified as lost in an accident only suffered this accident because of prior battle damage. Still, many more were lost in bona fide accidents (just compare the “total” and “in combat” numbers in the very same Wikipedia article you've linked).

However, the greatest threat to airplanes was posed by surface-to-air missiles (SAMs), and specifically the S-75 Dvina (NATO name SA-2 Guideline), which was also radar guided. According to Peter Davies,

Overall, fixed-wing US losses to SAMs amounted to 205 (against North Vietnamese claims of 1,293 kills) . . . Although this is a comparatively small proportion of the 3,322 losses (with 3,265 fatalities) to all causes, the effect of the SAM threat has to be measured in terms of its deterrence and disruption too. . . . In the two peak years of activity, SA-2 sites engaged US aircraft on 1,104 occasions in 1967 and 1,135 times in 1972. However, they too paid heavily. Of the 95 batteries of SA-2s (7,658 missiles) provided by the Soviet Union, only 39 remained in January 1973, with a little over 800 missiles available – some of which were unserviceable.

The point made by Davies about disruption is, in my opinion, the crucial factor brought about by the S-75. Quite simply, it forced American aircraft to descend below the S-75’s minimum altitude of engagement, i.e. about 3,000 meters—straight into the waiting hands of the AAA gunners. What North Vietnam had was an early attempt at building a multi-layered integrated air defense system, and for the most part, the attempt was successful.

And just as a final note, contrary to what pop culture likes to show, guided missiles don’t chase after planes as if we were living in a Looney Tunes cartoon. If one misses while flying at 3 times the speed of sound, it won’t be able to circle around for another go.