Was this already a hope from the earliest origins of Nazi Ideology (i.e. the 1920s) or was it the result of later development, perhaps not until nearly the very time it actually occurred? To what degree was the genocide the brainchild of Hitler alone, or was it perhaps many or most of the Nazi officials shared in this deliberate intention?
The question you're asking is at the heart of the most significant historiographic debate in the field of Holocaust studies, the debate between functionalism and intentionalism. The functionalist interpretation, generally speaking, is that Hitler created the political conditions that made the Holocaust possible, but that the Final Solution emerged as the result of a gradual process of radicalization that was primarily driven by lower-level actors within the Nazi hierarchy rather than being directly orchestrated by Hitler. Meanwhile, the intentionalist school argues that Hitler had a more or less concrete plan to carry out the Final Solution from the time that the Nazis came to power (or earlier, according to some intentionalists) and that the development of the Final Solution was the result of Hitler's direction. This was really the original debate in the historiography of the Holocaust as the field emerged in the 1960s and 1970s and (along with the Historikerstreit in Germany in the 1980s) played a pivotal role in shaping the field as it exists today.
The foundational work of Holocaust historiography was Raul Hilberg's The Destruction of the European Jews, which was published in 1961. Hilberg is generally regarded as the founder of the functionalist school and The Destruction of the European Jews as its foundational text. Hilberg's essential argument is that Nazi persecution of the Jews followed a pattern of gradual radicalization, from discrimination to exclusion to ghettoization to extermination, but that extermination wasn't the original plan or an inevitable result of the process. Later functionalist works stemmed from the arguments that Hilberg advanced, and included both strong and weak functionalist arguments. Among those who advanced a strong functionalist argument was Martin Broszat, who argued that the extermination of the Jews was initiated at the lower levels largely without input from the Nazi leadership, while others like Christopher Browning advanced a weaker functionalist argument, explaining that the Final Solution evolved because of the unstable political relationships between the various elements of the Nazi bureaucracy each trying to gain Hitler's favor by doing what they thought he wanted in relation to the "Jewish Question" (this process is sometimes referred to as "working toward the Führer" or the "bottom up" theory). Browning wrote arguably the seminal work on the role of lower-level Holocaust perpetrators, Ordinary Men, demonstrating that while the environment in which the Holocaust occurred was the result of political and military situations created by the Nazi leadership, lower-level actors took significant actions in shaping the actual killing process. The roles of these lower-level actors are well-documented in the historical record and borne out by eyewitnesses on the ground, so the factual aspects are relatively easy to prove empirically, even if their significance is a matter of interpretation.
There's not exactly an equivalent of The Destruction of the European Jews in the intentionalist school, but probably the seminal work was Lucy Dawidowicz's The War Against The Jews 1933-1945, published in 1975. She established what is basically the strong intentionalist argument, that Hitler not only planned the Final Solution, but that he had conceived of it before coming to power and even before writing Mein Kampf. Another strong intentionalist argument was presented by Daniel Jonah Goldhagen in his (extremely controversial) book Hitler's Willing Executioners, in which he argued that Germany was uniquely predisposed to an eliminationist policy toward the Jews even before the Nazis came to power in 1933. These arguments are, frankly, not taken that seriously anymore, but there are weaker intentionalist arguments like those presented by Saul Friedlander and others, who argued that there was a master plan created by Hitler, but only after the Germans came to power. These arguments are difficult to prove empirically because of the lack of a direct written order from Hitler for the extermination of the Jews and tend to rely on circumstantial evidence and interpretation of text rather than hard documentary evidence, since that doesn't exist.
As is usually the case when you have this kind of historical dialectic, there was ultimately a synthesis of the two theses. Generally speaking, the modern consensus is probably closest to the moderate functionalist argument, i.e. that Hitler was essential in creating the environment in which the Holocaust took place, but that the Final Solution was the result of a gradual process of radicalization at the lower levels, while still acknowledging the key role played by the Nazi leadership in guiding the overall process and that Hitler was the driving force even if there wasn't a pre-existing master plan and the Holocaust couldn't have happened without him. It's hard to say that a historical debate is ever "settled", but the extremes of the two schools of thought have largely been marginalized and the consensus is pretty well established at this point, even though new work continues to add nuance to the debate.
There has already been a ton of ink spilled on this question and I don't want to go on forever with excessive detail, but hopefully this gives you a good overview of how historians think about these things. Sorry that I can't give you a hard and fast answer, but unfortunately history doesn't really lend itself to simple answers to very complex questions.
Something that hasn't been mentioned is that the Nazis tried various other "solutions" to the "Jewish problem" before settling on the "final solution" of genocide. The first solution they tried happened in the first few years after they came to power in 1933, which was to make life so unpleasant for Jews in Germany that they would choose to emigrate. They were largely successful at the first part between the Nuremberg laws creating legal segregation and stripping Jews of full German citizenship, as well as harsh economic boycotts against Jewish businesses and bans on Jews being employed caused widespread destitution. The problem was there were few places for German Jews to emigrate to. Britain and the United States accepted barely any refugees. A great number of German Jews did manage emigrate to France, Belgium and the Netherlands prior to the outbreak of WWII. However, once war broke out, the rapid German successes on the Western front rendered this emigration "solution" largely moot, as many Jews who had emigrated from Germany were once again in German territory, and in addition numerous French, Belgian, Dutch and Polish Jews fell into German hands with the annexations of France, the low countries and Western Poland, and additional emigration to the US or UK was out of the question.
The next plan was to forcibly resettle European Jews in Madagascar, which was at the time a French colony and so easy for Germany to exert control over. However, this "solution's" feasability was contingent on Germany winning the Battle of Britain and forcing Britain out of the war, which would have granted Germany dominance on the high seas and hegemony in Africa, making the Madagascar plan workable. However, Germany was not successful in the Battle of Britain, foreclosing this option. It is important to note that this policy was part of the gradual escalation towards genocide, as the SS envisioned many of the resettled Jews dying due to the primitive and overcrowded living conditions that would result.
This failure, combined with the opening of the Eastern Front in Operation Barbarossa had a radicalizing effect on Nazi policies towards Jews. Unlike on the Western front, where Nazi racial ideology viewed the war goal as bringing fellow Aryan Europeans into the fold of the Reich, the Eastern Front was viewed as a war of extermination against the "Asiatic" and "subhuman" Slavs. Almost immediately upon the initial successes of the operation, SS Einsatzgruppen (along with a few divisions of drafted Wehrmacht military police) got to work behind the main lines of the German advance killing hundreds of thousands of Slavic as well as Jewish civilians in the conquered territories. Furthermore, with the initial German successes in the east, several million more Jews came under German control, only further magnifying the Nazis' "Jewish problem". It was at this point where the Nazis, "out of options", settled on a "Final Solution to the Jewish Question" of escalating from mass executions by shooting of Soviet Jews to the industrial murder of all Jews living in areas under German control or influence.
As this sequence of events makes clear, the Nazis did not immediately view genocide as necessary to solve their "Jewish problem". However, the failure of other "solutions", combined with the German successes in the first few years of WWII multiplying the problem, as well as the mass murders in the East in 1941 making genocide more normalized, led to a gradual radicalization that culminated in industrial murder. In this view, the Holocaust was not inevitable. Had the countries of the world been willing to resettle Polish and Central and Western European Jewry, or had Britain capitulated in 1940, the version of the Holocaust we are familiar with may not have occured.
Hi! As this question pertains to basic, underlying facts of the Holocaust, I hope you can appreciate that it can be a fraught subject to deal with. While we want people to get the answers they are looking for, we also remain very conscious that threads of this nature can attract the very wrong kind of response. As such, this message is not intended to provide you with all of the answers, but simply to address some of the basic facts, as well as Holocaust Denial, and provide a short list of introductory reading. There is always more than can be said, but we hope this is a good starting point for you.
##What Was the Holocaust?
The Holocaust refers the genocidal deaths of 5-6 million European Jews carried out systematically by Nazi Germany as part of targeted policies of persecution and extermination during World War II. Some historians will also include the deaths of the Roma, Communists, Mentally Disabled, and other groups targeted by Nazi policies, which brings the total number of deaths to 11-17 million. Debates about whether or not the Holocaust includes these deaths or not is a matter of definitions, but in no way a reflection on dispute that they occurred.
##But This Guy Says Otherwise!
Unfortunately, there is a small, but at times vocal, minority of persons who fall into the category of Holocaust Denial, attempting to minimize the deaths by orders of magnitude, impugn well-proven facts, or even claim that the Holocaust is entirely a fabrication and never happened. Although they often self-style themselves as "Revisionists", they are not correctly described by the title. While revisionism is not inherently a dirty word, actual revision, to quote Michael Shermer, "entails refinement of detailed knowledge about events, rarely complete denial of the events themselves, and certainly not denial of the cumulation of events known as the Holocaust."
It is absolutely true that were you to read a book written in 1950 or so, you would find information which any decent scholar today might reject, and that is the result of good revisionism. But these changes, which even can be quite large, such as the reassessment of deaths at Auschwitz from ~4 million to ~1 million, are done within the bounds of respected, academic study, and reflect decades of work that builds upon the work of previous scholars, and certainly does not willfully disregard documented evidence and recollections. There are still plenty of questions within Holocaust Studies that are debated by scholars, and there may still be more out there for us to discover, and revise, but when it comes to the basic facts, there is simply no valid argument against them.
##So What Are the Basics?
Beginning with their rise to power in the 1930s, the Nazi Party, headed by Adolf Hitler, implemented a series of anti-Jewish policies within Germany, marginalizing Jews within society more and more, stripping them of their wealth, livelihoods, and their dignity. With the invasion of Poland in 1939, the number of Jews under Nazi control reached into the millions, and this number would again increase with the invasion of the Soviet Union in 1941. Shortly after the invasion of Poland, the Germans started to confine the Jewish population into squalid ghettos. After several plans on how to rid Europe of the Jews that all proved unfeasible, by the time of the invasion of the Soviet Union in 1941, ideological (Antisemitism) and pragmatic (Resources) considerations lead to mass-killings becoming the only viable option in the minds of the Nazi leadership. First only practiced in the USSR, it was influential groups such as the SS and the administration of the General Government that pushed to expand the killing operations to all of Europe and sometime at the end of 1941 met with Hitler’s approval.
The early killings were carried out foremost by the Einsatzgruppen, paramilitary groups organized under the aegis of the SS and tasked with carrying out the mass killings of Jews, Communists, and other 'undesirable elements' in the wake of the German military's advance. In what is often termed the 'Holocaust by Bullet', the Einsatzgruppen, with the assistance of the Wehrmacht, the SD, the Security Police, as well as local collaborators, would kill roughly two million persons, over half of them Jews. Most killings were carried out with mass shootings, but other methods such as gas vans - intended to spare the killers the trauma of shooting so many persons day after day - were utilized too.
By early 1942, the "Final Solution" to the so-called "Jewish Question" was essentially finalized at the Wannsee Conference under the direction of Reinhard Heydrich, where the plan to eliminate the Jewish population of Europe using a series of extermination camps set up in occupied Poland was presented and met with approval.
Construction of extermination camps had already begun the previous fall, and mass extermination, mostly as part of 'Operation Reinhard', had began operation by spring of 1942. Roughly 2 million persons, nearly all Jewish men, women, and children, were immediately gassed upon arrival at Bełżec, Sobibór, and Treblinka over the next two years, when these "Reinhard" camps were closed and razed. More victims would meet their fate in additional extermination camps such as Chełmno, but most infamously at Auschwitz-Birkenau, where slightly over 1 million persons, mostly Jews, died. Under the plan set forth at Wannsee, exterminations were hardly limited to the Jews of Poland, but rather Jews from all over Europe were rounded up and sent east by rail like cattle to the slaughter. Although the victims of the Reinhard Camps were originally buried, they would later be exhumed and cremated, and cremation of the victims was normal procedure at later camps such as Auschwitz.
##The Camps
There were two main types of camps run by Nazi Germany, which is sometimes a source of confusion. Concentration Camps were well-known means of extrajudicial control implemented by the Nazis shortly after taking power, beginning with the construction of Dachau in 1933. Political opponents of all type, not just Jews, could find themselves imprisoned in these camps during the pre-war years, and while conditions were often brutal and squalid, and numerous deaths did occur from mistreatment, they were not usually a death sentence and the population fluctuated greatly. Although Concentration Camps were later made part of the 'Final Solution', their purpose was not as immediate extermination centers. Some were 'way stations', and others were work camps, where Germany intended to eke out every last bit of productivity from them through what was known as "extermination through labor". Jews and other undesirable elements, if deemed healthy enough to work, could find themselves spared for a time and "allowed" to toil away like slaves until their usefulness was at an end.
Although some Concentration Camps, such as Mauthausen, did include small gas chambers, mass gassing was not the primary purpose of the camp. Many camps, becoming extremely overcrowded, nevertheless resulted in the deaths of tens of thousands of inhabitants due to the outbreak of diseases such as typhus, or starvation, all of which the camp administrations did little to prevent. Bergen-Belsen, which was not a work camp but rather served as something of a way station for prisoners of the camp systems being moved about, is perhaps one of the most infamous of camps on this count, saw some 50,000 deaths caused by the conditions. Often located in the Reich, camps liberated by the Western forces were exclusively Concentration Camps, and many survivor testimonies come from these camps.
The Concentration Camps are contrasted with the Extermination Camps, which were purpose built for mass killing, with large gas chambers and later on, crematoria, but little or no facilities for inmates. Often they were disguised with false facades to lull the new arrivals into a false sense of security, even though rumors were of course rife for the fate that awaited the deportees. Almost all arrivals were killed upon arrival at these camps, and in many cases the number of survivors numbered in the single digits, such as at Bełżec, where only seven Jews, forced to assist in operation of the camp, were alive after the war.
Several camps, however, were 'Hybrids' of both types, the most famous being Auschwitz, which was a vast complex of subcamps. The infamous 'selection' of prisoners, conducted by SS doctors upon arrival, meant life or death, with those deemed unsuited for labor immediately gassed and the more healthy and robust given at least temporary reprieve. The death count at Auschwitz numbered around 1 million, but it is also the source of many survivor testimonies.
##How Do We Know?
Running through the evidence piece by piece would take more space than we have here, but suffice to say, there is a lot of evidence, and not just the (mountains of) survivor testimony. We have testimonies and writings from many who participated, as well German documentation of the programs. This site catalogs some of the evidence we have for mass extermination as it relates to Auschwitz. I'll end this with a short list of excellent works that should help to introduce you to various aspects of Holocaust study.
##Further Reading