I've heard people list off "facts" such as there's no signed documents by Hitler ordering the final solution.

This much, at least, is true. There is no document out there to the effect of "systematically murder all Jews. Yours, Hitler". The lack of such a document is essentially the crux of an ongoing and valid conversation among historians of WWII, the Reich, and Hitler as to how much individual responsibility Hitler carries for the Holocaust. Ron Rosenbaum argues in his book, Explaining Hitler (incidentally an excellent read for all issues about understanding who Hitler was and why he did what he did) that the lack of such a document is essential to the popular fascination with Hitler and his persistence as the embodiment of evil. We convince ourselves that if such a document existed, then maybe we could understand why Hitler did the things he did and was who he was - not empathize, but maybe understand. This is tremendously more appealing to most people than the alternative: that we cannot understand, that there was no reason that would make sense, that ultimately 6 million people died for no reason at all.

But I digress. The logistical details of the Final Solution were hammered out at the Wannsee Conference, attended by top Nazi officials with the purpose setting out "the complete plan for the extermination of the European Jewry" (see Gerlach). Hitler did not attend this meeting and the proceedings were largely carried out by SS bigwig Reinhard Heydrich. Adolf Eichmann's minutes of this meeting, preserved by Martin Luther in direct contrast to the behaviour of virtually every other attendee of Wannsee (and, IIRC, in defiance of orders), are the only record of the meeting, by far the clearest source on an institutional campaign to eliminate the Jews we have, and even that is a heavily edited document by Eichmann's own admission.

This is ultimately at the heart of why there is no "Please kill all Jews, regards from the Fuhrer" letter or document out there (much, as Rosenbaum writes, as we may wish to one day find one hidden away in some safe deposit box in a Swiss bank somewhere). The Nazis knew exactly what it looked like to have your name attached to a document that ordered and authorized genocide and what it would mean if things ever went south for any of them, let alone for the entire German war effort. As a result, there was concerted effort among top Nazi officials, especially as the war started to go badly for them, to distance themselves from the camps and the Holocaust. The concerted destruction of almost every copy of Eichmann's minutes from the Wannsee Conference is an example of this.

Note that the fact that Hitler was absent from the Wannsee Conference is also rendered essentially meaningless by this fact. It has been argued that the Conference's real purpose was for Heydrich to assert control over the operational structure of the Final Solution, and that it happened directly as a result of a decision made consciously by Hitler about how to pursue the Final Solution in 1941 (again see Gerlach). Be wary of claims that Hitler knew nothing about the Holocaust. His precise role in it is still very much an open discussion, but denying that he had any knowledge of it has the dubious honour of being the first move toward the fringe by a number of notorious Holocaust deniers.

Sources:

• Explaining Hitler, Ron Rosenbaum

• "The Wannsee Conference, the Fate of German Jews, and Hitler's Decision in Principle to Exterminate All European Jews", The Journal of Modern History 70.4, Christian Gerlach. Note that this resource is a little dated, but it's a good scholarly overview of Wannsee from a reputable journal that's available for free.

• The Origins of the Final Solution, Christopher R. Browning

And while I didn't consult it for this post, Ian Kershaw's biography of Hitler is always a first-rate source on all things Hitler that's well worth a good scouring and regular referring back to.