Albert Einstein certainly sealed the current image, but as a trope the idea extends back into antiquity; the most famous example being Archimedes, as related by the Roman writer Marcus Vitruvius Pollio about the matter of the Golden Crown:

In the case of Archimedes, although he made many wonderful discoveries of diverse kinds, yet of them all, the following, which I shall relate, seems to have been the result of a boundless ingenuity. Hiero, after gaining the royal power in Syracuse, resolved, as a consequence of his successful exploits, to place in a certain temple a golden crown which he had vowed to the immortal gods. He contracted for its making at a fixed price, and weighed out a precise amount of gold to the contractor. At the appointed time the latter delivered to the king’s satisfaction an exquisitely finished piece of handiwork, and it appeared that in weight the crown corresponded precisely to what the gold had weighed.

But afterwards a charge was made that gold had been abstracted and an equivalent weight of silver had been added in the manufacture of the crown. Hiero, thinking it an outrage that he had been tricked, and yet not knowing how to detect the theft, requested Archimedes to consider the matter. The latter, while the case was still on his mind, happened to go to the bath, and on getting into a tub observed that the more his body sank into it the more water ran out over the tub. As this pointed out the way to explain the case in question, he jumped out of the tub and rushed home naked, crying with a loud voice that he had found what he was seeking; for he as he ran he shouted repeatedly in Greek, “Εὕρηκα, εὕρηκα” [= “Eureka, eureka” = “I have found (it), I have found (it)”].

The idea of a soaking wet and naked philosopher streaking through Syracuse sort of nails the whole "absent-minded genius" trope; and the image for Archimedes in particular may have been strengthened by another apocryphal episode, concerning his death:

But as Archimedes was drawing diagrams with mind and eyes fixed on the ground, a soldier who had broken into the house in quest of loot with sword drawn over his head asked him who he was. Too much absorbed in tracking down his objective, Archimedes could not give his name but said, protecting the dust with his hands, “I beg you, don’t disturb this,” and was slaughtered as neglectful of the victor’s command; with his blood he confused the lines of his art. So it fell out that he was first granted his life and then stripped of it by reason of the same pursuit.