The 'famed' attack of emperor Caligula against the sea is something of a modern take on an incident that the roman historian Suetonius wrote in his De Vita Caesarum (The Life of the Caesars) roughly 50-60 years after Caligula's death.
As read here:
Finally, as if he intended to bring the war to an end, he drew up a line of battle on the shore of the Ocean, arranging his ballistas and other artillery; and when no one knew or could imagine what he was going to do, he suddenly bade them gather shells and fill their helmets and the folds of their gowns, calling them "spoils from the Ocean, due to the Capitol and Palatine." As a monument of his victory he erected a lofty tower, from which lights were to shine at night to guide the course of ships, as from the Pharos. Then promising the soldiers a gratuity of a hundred denarii each, as if he had shown unprecedented liberality, he said, "Go your way happy; go your way rich."
Neither Suetonius nor the works of the greek senator Cassio Dio of 229 A.D and Sextus Aurelius Victor in 361 A.D which are the three primary sources we have on this explicitly states that this is taken as a 'war' against the gods, or of Neptune in particular.
For context: The story depicted was intended to illustrate an event that happened when Caligula rode with two legions to the shores of the (now) British channels around roughly 40 A.D. It's not entirely certain what his purpose was there, though many would say the reasoning was to cross the channel and conquer the British isles.
The story of Caligula's ordering his legions to pick up these seashells has in modern history divided scholars into roughly two large camps:
There are those that accepts the account and agree that this order was most likely given (though the reasoning, there are many which I'll touch upon later)
Later chronicles of Caligula's life is seriously corrupted and this story, as with many other anecdotes of his life, was meant to paint him as a crazed and dangerous man.
The former camp does not necessarily subscribe to the idea that Caligula was as mad as historians like Suetonius and Cassio would have us believe. He was very likely a reviled man by the senate and nobility, many of his enactments were unpopular and there are serious indications that he did at least try to implement himself as a Living God after the fashion of greek kings. This was controversial and one must remember that this was a huge issue of the first Emperor of the Roman Empire, Augustus, who many were ready to worship him as a god-king (and many ready to condemn him for trying) but always managed to dissuade outright worship.
Theories of why he ordered his soldiers to pick up seashells have been many; either as a madman's ravings, attempt to humiliate his two legions who refused to board the boats to invade Britain (as the weather was reportedly rather bad) or as a symbolic gesture of his mastery of the ocean.
The latter who believe this did not happen would argue that this as either as propaganda by Suetonius and later historians, or a matter of serious mistranslation from the manuscripts of much later dates which we might have them. I wont go into too much detail, but the latin word concha can have several meanings including mollusc or shellfish, but also "objects shaped like a shell" which some could argue could be taken to mean "small boat" and that in fact Caligula did not order the gathering of seashells but the boats to be paraded in his triumph.
In any case, the story itself is just one of many others of his supposed madness and this in its stead did not have too much of an impact on his rule. It is far more likely that his ideas of a more eastern and idea of a divine king was what truly made his rule controversial and contentious, as was his many attempts to curtail the nobility and the senate.
A lot of the history of the roman emperors have quite naturally come at a later date than that of the emperor and his reign. There are many indications that these stories that have been in circulation and to which the roman historians published were not necessarily or readily believed. Another example is that of the syrian priest-king Heliogabalus who reigned as emperor of Rome and tried to make the syrian god Elagabal (Sol Invictus Elagabal) the 'King of the Gods' -- his rule was short lived and he was murdered by his own Praetorian Guard and his memory banished (damnatio memoriae). What we know of him was published at a much latter date in the Historia Augusta and even in these texts we find the chroniclers doubting whether or not the sordid and quite shocking tales of the boy-emperor Heliogabalus could be true.
I hope this answered in part what you was curious about!
Sources
Suetonius, The Lives of the Caesars
Retrieved from: http://penelope.uchicago.edu/Thayer/E/Roman/Texts/Suetonius/12Caesars/Caligula*.html
Woods, D. (2000). Caligula's Seashells. Greece & Rome, 47(1), 80-87.
Winterling, A. (2011). Caligula : A biography.