To understand the premise and scope of Impressionism, one has to understand the environment of the society in which it was born. The major (and in essence the only) venue in France for an artist to gain notoriety, exposure, and by extension financial promise was the Salon de Paris. This was a annual or biannual juried exhibition of the Académie des Beaux-Arts. Up through the 1860s and 70s the Salon was royally sponsored, and filled with art of the 'academic' style (see Bougereau). This style was celebrated for its technical complexity—employing the use of blending, the concealment of brushwork, and careful tonal gradation. The conservatism of this aesthetic stems from its similarities to the earlier Neo-classical and Mannerist styles. Simply put, the 'academic' style was accessible, mechanically impressive, and familiar. It fit the audience's preconceived notions of the purpose and look of art.

But one day in 1863, the Salon jury headed by Le Comte de Nieuwerkerke refused an unforgivably high number of paintings. Among these artists refused were Manet and Whistler, who would later come to be considered the fathers of impressionism. But to put it into context, over 3,000 works total were refused, and an uproar ensued. In an effort to appear diplomatic, Napoleon III opened a new exhibition where all the refused works would be shown—fittingly called the 'Salon des Refusés'. This exhibition was a massive object of ridicule for the public, but nonetheless, much more successful than the actual salon exhibition itself. Emile Zola, the avant-garde writer and art critic, reported how visitors, over 1,000 a day, shoved and pushed their way in to laugh at this new modern art.

By 1874, the Impressionists began to exhibit their works in their own exhibitions held at Nadar's studio. In a review, Louis Leroy wrote sardonically, "'Ah! This is it, this is it!: he cried in front of n. 98. 'This one is Papa Vincent's favorite! What is this a painting of? Look in the catalogue.' 'Impression, Sunrise.' 'Impression-- I knew it. I was just saying to myself, if I'm impressed, there must be an impression in there… And what freedom, what ease in the brushwork! Wallpaper in its embryonic state is more labored than this seascape!". But it would be incorrect to assume there was no early praise for this new group of artists. Theodore Duret, in a review, wrote that Monet was "no longer painting merely the immobile and permanent aspect of a landscape, but also the fleeting appearances which the accidents of atmosphere present to him, Monet transmits a singularly lively and striking sensation of the observed scene".

An accurate picture of the public response to their new art is typified in a 1876 weekly publication:

"The rue le Peletier is a road of disasters. After the fire at the Opera, there is now yet another disaster there. An exhibition has just been opened at Durand-Ruel (N.B.: an art dealership) which allegedly contains paintings. I enter and my horrified eyes behold something terrible. Five or six lunatics, among them a woman, have joined together and exhibited their works. I have seen people rock with laughter in front of these pictures, but my heart bled when I saw them. These would-be artists call themselves revolutionaries, "Impressionists". They take a piece of canvas, colour and brush, daub a few patches of paint on it at random, and sign the whole thing with their name. It is a delusion of the same kind as if the inmates of Bedlam picked up stones from the wayside and imagined they had found diamonds."

The reviewer's use of the word 'lunatics', and the description of viewers rocking in laughter seem especially similar to popular reception of much contemporary art.

Your description of artists as being seen as 'kind of crazy' stems from a movement of European bohemianism which started in France in the early 19th century. Certainly some of the impressionists suited this lifestyle in the earlier years. It's probably important to qualify the type of crazy that you mean, because in contrast to the early modernists of say Toulouse-Lautrec or Picasso, the Impressionists certainly lived less 'seedy' lives. Most of them were married, and while they often expressed radical political views, the subject matter of their paintings is usually quite bourgeois and polite. It is expressly clear that the average individual, unattuned to the developments of art in the years directly proceeding Impressionism with individuals such as Courbet and members of the Barbizon school, would feel threatened and confused by this new style. The artistic intention of the Impressionists didn't conform with classical precedents, and just as contemporary art does today; it defied the public's idea of what constitutes good art. It is understandable that to the uninitiated, it would seem that these artists were 'crazy' for purposefully creating what seems to be 'bad art'.

Sources: E.H. Gombrich, The Story of Art, (London, 1995)

Rewald, History of Impressionism, (New York, 1973)

Sieberling, Impressionism, (Oxford, 1996)

edit: grammar.