This is quite a complicated question. Was cavalry really small and insignificant in Classical Greek warfare?

First we have to narrow down the Greece you're talking about. In the late Archaic and early Classical period, the Greek and semi-Greek communities of Macedon, Thessaly, Boiotia, Thrace, Chalkis, Eretria, Kolophon, Magnesia, Lydia, Sicily and Southern Italy are all known or assumed to have fielded effective cavalry. Iain Spence (in The Cavalry of Classical Greece (1993)) claims that Lokris and Phokis also raised a cavalry force from the 470s BC onwards. The Athenians had a small corps of 100 cavalry throughout the 6th century BC, but they also relied on their Thessalian allies to provide horsemen, removing the need to organise their own. When the Thessalians betrayed them in 457 BC, they promptly formed their own cavalry corps. If we add all this together, we're left with quite a small bit of Greece that seems to have used no cavalry - namely, the Peloponnese. That's it.

We also have to narrow down the time you're talking about. Troop types don't seem to have fought separately in homogenous units until the late 6th century BC, so there's no point talking about a lack of cavalry before that time. Meanwhile, Sparta organised a cavalry force in 425/4 BC, and by the 4th century BC every state on the Peloponnese had its own cavalry. So when we say there is a period in which cavalry was not used by the Peloponnesians, we're really only talking about the very late Archaic and early Classical period (say 550-400 BC). Before this, the term "cavalry" is meaningless, and after it, everyone used cavalry.

These cavalry forces weren't small, either. At the height of their power, the Athenians could raise a citizen levy of about 13,000 hoplites and 1,000 cavalry. While the full levy of infantry was very rarely called up, the cavalry was supposed to be constantly at the state's disposal. The proportion in other states is even higher. Syracuse, which was perhaps as big as Athens in 415 BC, could field 1,200 horsemen; the militia of the Boiotian League in the early 4th century had a paper strength of 11,000 hoplites and 1,100 cavalry, while the Spartan army of the mid-4th century consisted of approximately 3,400 hoplites and 720 cavalry. Most impressively, Xenophon claims (Hellenika 6.1.19) that the Thessalian tyrant Iason could levy a force of 20,000 hoplites and 8,000 cavalry. These numbers are mind-bogglingly vast. In fact, at both Athens and Sparta they exceeded the total number of people who could actually afford to equip themselves as cavalry. Their mounted forces had to be subsidized by the state. We are told (Xenophon, The Cavalry Commander 1.19) that the Athenian state in the 4th century BC paid as much as 40 talents a year for the privilege of having an effective cavalry.

So, what we actually see is that all Greek states invested in fielding as large a cavalry force as they could afford, except for the states of the Peloponnese in the century or so between the middle of the 6th and the middle of the 5th century BC. What we really need to ask, then, is why they didn't use cavalry during that time, bucking the trend of the entirety of Greek history.

There are two common explanations. The first is that using cavalry was just not Greek; a proper Greek citizen fought as a hoplite, and mounted warfare was for rich showy cowards. There is some evidence that the fighting style of the horseman was considered easier than that of the hoplite, but, as I hope the list above shows, the overall point is nonsense. Many Greek peoples were famous for their horsemen from the time these first became a thing. Most Greek states were effectively ruled by the horse-owning class. The tactical importance of the horse was widely acknowledged, and the vulnerability of hoplites to cavalry (as you point out) was one of the forces that shaped Greek warfare as a whole. Hoplites simply could not afford to be snobbish about horsemen; they would be immediately and thoroughly trounced by any mounted force they dared to underestimate. Having their own cavalry to ward off enemy mounted forces was essential to their survival.

The second common explanation is that Greece just isn't suitable for raising horses: the land was too poor, there wasn't enough pasturage, and not enough people could afford to keep horses to raise a proper cavalry force from their ranks. There is certainly truth in this; something like 80% of Greece is mountainous, and no one is surprised to find that the people of remote regions like Arkadia or Aitolia did not fight as cavalry. Conversely, plains like Boiotia and Thessaly were the home of famously numerous and powerful cavalry. The fact that the region that did not use cavalry seems to overlap with the most rugged part of Greece shows that this point must go a long way toward answering your question.

However, it does not answer it in its entirety. Sparta, for instance, was situated in a relatively large, relatively flat area, and its inhabitants were famously keen on horses. They are frequently recorded to have won the chariot race at the Olympic Games. Their elite unit was called the hippeis - horsemen. So why didn't they fight as cavalry? And for that matter, why wouldn't rich cities like Corinth, Elis and Argos be able to field their own, given that they all eventually did?

I am kind of at a loss here. No source explains why they didn't use cavalry during the late Archaic and early Classical period. I could argue that the terrain wouldn't allow it, but later cavalry warfare in the Peloponnese gives the lie to this assumption. Really, I can only guess.

So I will!

Both Worley (Hippeis (1994)) and Sidnell (Warhorse (2012)) have noted that the rise of true cavalry in the late 6th century BC was suddenly stunted by the rise of the hoplite phalanx at roughly the same time. As a counter-cavalry tactic, the phalanx was so effective that it briefly marginalised cavalry altogether, until horsemen could find a way to adapt to the new combat situation. Peoples whose social structure was built around a ruling cavalry elite were quicker to find solutions than states based on a broader hoplite levy with a small leisure class, but eventually the use of cavalry spread from the north back into mainland Greece until all Greek city-states were once again on board the combined arms train. Sparta, as a society built around outward equality, had the most trouble changing its hoplite ways; creating a cavalry would establish a class distinction that the Spartan state was specifically designed to remove. But by the Peloponnesian War even the Spartans realised that cavalry was essential to the tactics of their age.

Followup: what are the logistics behind supply lines for cavalry vs infantry in Greek armies of the era?