First, a tricky misconception: Einstein didn't actually persuade the US to start the Manhattan Project. He persuaded (in a way) Roosevelt to allocate a very modest amount of government resources to looking into the question of whether nuclear weapons were feasible, in 1939. The actual Manhattan Project did not start until 1942, entirely independent of Einstein. There is an entirely separate story there; it was not a straightforward matter of "convincing," and the people who did push for the bomb production program underestimated its total cost by a significant factor.

Second, as for Von Braun, et al. German engineers had been enthusiastic about rocketry since before the Nazis came to power. Several engineers who had been involved in the Verein für Raumschiffahrt, an amateur rocket society, ended up going and working for the German Army after the Nazis came to power. Von Braun was working for the Wehrmacht as early as 1933, designing what would eventually become the V rockets. Work on what would become the V-2 started as early as 1936, with small-scale models. Walter Dornberger, head of the rocket program, pushed for production in 1939, but was turned down, because Hitler was dubious of the expense and utility.

Why, and what changed? Remember that the big Blitzkrieg plan was to quickly take Poland, then France, then turn around and take the USSR, and then focus in on the UK. This was not a siege mentality — it was about rapid conquest. But in the winter of 1941 the German offensive ran into the Russian winter and the blitz was halted. The US also entered the war, portending a different sort of approach.

Over the course of the fall of 1942, Hitler's mind changed, and in early 1943, he agreed to allocate the V-2 program the resources necessary for production (but they were not ready to use until 1944 — many changes had to be put into place to make the original design work). The appeal of the rockets to Hitler was twofold: 1) they would keep up the attack against the UK without requiring the potential sacrifice of manned craft, and 2) they were impossible to defend against militarily, and they thought (per the strategic doctrine at the time, shared by both the Axis and Allies) that if the civilian populace felt completely terrorized by aerial weapons they would sue for peace. (In practice, that didn't work for either side.) He also was afraid that the US would employ similar weapons against the Germans. (There is some irony there: the US Manhattan Project was formed to "race" a non-existent German bomb project, and the German V-2 program was formed, in part, to "race" a non-existent American long-range rocket project.)

Lastly, it is worth noting that rarely do leaders approve these sorts of megaprojects knowing what the actual costs will be. With both the Manhattan Project and the V-2 program, a "production" program was approved that was pretty expensive but not ridiculous in the course of war projects. As it went on, it cost more and more money. It is easy to get into a situation where one feels that if you do not continue funding the project, the other funds already spent will have been wasted. Thus you get a "lock-in" effect, whereby you end up spending what in retrospect might have been more than what you expected to spend by a serious fraction. The Manhattan Project cost about 4X more than they expected it to in 1942. I don't know whether there was ever a total project V-2 cost (they fluctuated in their planning as to how many missiles to produce), but if it was anything like most big military projects, they likely didn't realize what it was going to cost them. Full costs of cutting-edge systems are hard to assess, even today.

Note that it was not von Braun, but Dornberger, who was actually administratively in charge of the project, and Dornberger was the one who was more crucial in convincing the leadership to support it.

Lots of details in Michael J. Neufeld, The Rocket and the Reich: Peenemunde and the Coming of the Ballistic Missile Era, chapter 5. By