Was this just a German problem, or was the machine generally flawed?
The F-104's reputation is a complicated story. The F-104 was, at its core, designed as the ultimate fighter for the Korean War - optimizing for speed, thrust-to-weight ratio, climb rate, and high altitude performance. The result was an excellent fighter for that role, but US Air Force requirements would move away from the aircraft Lockheed had designed. Performance came at the cost of technical complexity and range, and the result was that the fighter was almost a point-defence interceptor in its original incarnation. Though comparable in performance to the Soviet MiG-21, differences in doctrines meant that the F-104 wasn't able to keep up with other USAF Century-series fighters - aircraft like the F-101 and F-102, while slower, made up for their lower performance with their longer endurance and advanced avionics, and fighters like the F-100 and F-105 offered better flexibility and performance in strike and tactical fighter roles. Fewer than 300 examples were purchased, and they were largely relegated to interceptor roles before being phased in the aftermath of poor performance during a deployment to Vietnam in 1967.
Fortunately for Lockheed, their work on the F-104 wasn't a complete waste. In 1957, the new West German Air Force issued a tender for a single multirole aircraft to fill fighter, strike, and reconnaissance roles. Lockheed heavily reworked the F-104 - a new engine and larger tail were added, the blown flaps were improved for low-speed operations, and a new avionics suite complete with an advanced inertial navigation system was added. Though its 3,000lb payload was underwhelming, it was enough to carry the 2,000lb nuclear warhead the Germans intended to use, and despite its design optimized for high-speed, high-altitude flight, the rework for low-level strike wasn't as poorly thought-out as one might expect, as the design was fairly stable in high-speed flight at low levels.
Lockheed would eventually win this lucrative deal and would be accepted into service by not just West Germany, but much of NATO (as well as Japan). But allegations of bribery soon surfaced - allegations that would continue to follow Lockheed into the 1970s. It would eventually come out that Lockheed had paid $22 million in bribes to West German officials to get the F-104 accepted along with further alleged bribes and aggressive lobbying to other governments like Japan. That alone left a sour taste for many and created an easy thing to point to when problems arose.
And oh boy did problems arise. The F-104G - as the German variant was known - suffered from a host of problems in service. Much of it was technical, since the extreme performance the fighter offered required Lockheed to leverage some less mature technologies. The J79 engine was a major source of issues, causing the USAF to ground its examples due to engine-failure-related crashes early in the career, and these problems only got worse with the introduction of a more advanced version of the J79 on the F-104G. Engine teething problems would eventually be solved and the J79 would go on to be a successful engine in a number of other prominent designs like the F-4 Phantom, but its early performance did much to tarnish the reputation of the F-104. Compounding the problem of the engine issues was the aircraft's design optimized purely for high speeds. Even with the complex blown flaps, the F-104 was infamous for performing very poorly at low speeds and having a very high landing speed. So an engine failure was pretty obviously a critical issue in a single-engined aircraft with little weight behind it to keep up its momentum and poor performance at low speeds.
And that's a good way to segue into the operation-specific problems with the F-104. The Germans (as with many other operators) procured the F-104 as a nuclear strike fighter. Per the doctrine of the time, the F-104 was supposed to fly in to target areas at high speeds at very low altitudes, pitching up at a certain pre-determined distance from the target and releasing its payload as it climbed. Looping back and peeling away to escape at low altitude, the climbing release of the bomb would "toss" the payload some distance and allow the fighter to avoid direct overflight of the target and get some distance before the warhead detonated. As you can probably gather from this description, this is an incredibly dangerous proposition, even in good conditions. The F-104G had advanced avionics intended to help navigation at low levels, but again being on the bleeding edge of technology created a lot of risk when things inevitably failed. In Germany's case, many of their crashes were a result of these systems failing, with aircraft lost in what's known as Controlled Flight Into Terrain - the pilot crashes because they have no idea they're headed right into the side of a hill.
Adding to all of that was the unique situation West Germany found itself in during the early 1960s. The F-104 was a bleeding edge design when introduced, and the West German military hadn't been reformed until 1955. A lot of their institutional knowledge in terms of maintenance and operations of aircraft had been lost after WW2, and that which remained was centered around now-obsolete piston-engined aircraft and very early jets. Inadequate training can partially be blamed for the high crash rates, as poorly trained crews weren't able to service an already troublesome aircraft as well as they needed to. On top of that, the different flight characteristics were a far cry from what German pilots were used to in a fighter (Erich Hartmann famously decried it as a poor fighter). In short, the new aircraft was a dangerous proposition even for established forces like the USAF, so a force like Germany in the process of rebuilding its institutional knowledge was bound to suffer.
While the F-104 ends up getting a lot of flak (metaphorical, fortunately) for its high crash rate in German service, it's difficult to determine how much of that was the aircraft's fault itself. While troublesome for sure, the West German F-84F fleet suffered a higher attrition rate than the F-104Gs over their 11 years of service (36% of ~550 were lost), and those were flown in similar roles to the F-104Gs.