The initial source for that figure is believed to come out of the testimony of Wilheml Hoettl during the Nuremberg trials. You can read the relevant part of his testimony on this website or in any transcript from the trials. There he states that this was the figure Eichmann believed to be correct. This figure was brought up again in the Eichmann trial where the chief prosecutor referred to '6 million co-accusers'. The site I linked above has the full trial transcripts and the US Holocaust Memorial Museum has a bunch of video and documents related to it as well.

In Hilberg's "The Destruction of the European Jews" he comes up with a figure of 5.1 million (this was initial publishing of 1961). Other historians and scholars have come up with slightly varying figures using the primary sources available, but the range is generally believed to be 5-6 million Jewish victims.