It gets even more interesting - Plato's Socrates never said that!
Here's what Plato has him say about the oracle:^(1)
You know the kind of man Chaerephon was, how impetuous in whatever he undertook. Well, once he went to Delphi and made so bold as to ask the oracle this question - and, gentlemen, don't make a disturbance at what I say - for he asked if there were anyone wiser than I. Now the Pythia replied that there was no one wiser.
Socrates does respond modestly:
When I heard this, I thought to myself: “What in the world does the god mean, and what riddle is he telling? For I know full well that I am not very wise, or even a little wise. (translation heavily adapted from Perseus)
And goes on to investigate the oracle's claim by interrogating a well-known (but unnamed) politician, apparently thinking that he will be a wise man, only to find that he is actually a fool:
As I went away, I thought to myself, “I am wiser than this man; for neither of us really knows anything fine and good, but this man thinks he knows something when he does not, whereas I, as I do not know anything, do not think I do either. I seem, then, in just this little thing to be wiser than this man at any rate, that what I do not know I do not think I know either.”
This is close - here Socrates' wisdom consists in not thinking that he knows what he really doesn't. Indeed, at various points in Plato's corpus, he claims (usually a little disingenuously) not to know about the topic under discussion. However, this is quite different from claiming to know nothing. Indeed, at various points in the Apology (and elsewhere in the Platonic corpus), Socrates does claim to know something. Indeed, only a few sections later - a few minutes, if we treat the Apology as a speech - he claims to be so certain in his knowledge of the truth and his own moral compass that he is willing to die for it:
Men of Athens, I respect and love you, but I shall obey the god rather than you, and while I live and am able to continue, I shall never give up philosophy or stop exhorting you and pointing out the truth to any one of you whom I may meet.
Gail Fine wrote an article on this in 2008, as part of which she searched the Thesaurus Linguae Graecae for all instances of 'all I know is that I know nothing' and similar phrases.^(2) Metrodorus of Chios, Socrates' approximate contemporary, did come out with a superficially similar sentiment:^(3)
None of us knows anything, not even this, whether we know or do not know
However, she didn't find the idea of 'knowing only that you know nothing' in any ancient Greek text, and the closest she could find in relation to Socrates was this remark by Diogenes Laertius:
εἰδέναι μὲν μηδὲν πλὴν αὐτὸ τοῦτο [εἴδεναι]<h e said that> he knew nothing except [that he knew] just that
The square brackets are important - in a scholarly text they indicate something that has been transmitted in the manuscripts but which the editor believes was not part of the original.
Without that part, Fine points out that it's ambiguous - 'just that' could mean 'that he knows nothing', but more straightforwardly means 'nothing' - echoing perhaps his doubling-down modesty in the Apology 'I am not very wise, or even a little wise'. What this suggests is that a scribe at some stage in the text's transmission had a copy in front of him without 'that he knew' and considered that he ought to add it in. So we're now at an interesting question - by this point, the idea that Socrates 'knew that he knew nothing' must be out there. And yet, as far as we can tell, it doesn't come from Greek.