I've looked previously into references to fossils of dinosaurs and megafauna in ancient Greek and Roman literature for fun. There's actually quite a lot of passages that seem to concern remains of pre-historical animals in Classical literature; some hundred references in about thirty authors. Which is not that surprising, since as far as I understand (warning - historian trying to talk natural sciences), from about 23 million years to 12,000 years ago, the Mediterranean basin was not a sea, but a land mass populated by a wide variety of prehistoric mastodons, rhinoceroses, cave bears, and other massive beasties. Fossils of pre-historic giants are pretty common around the Mediterranean and new fossils crop up now and then due to erosion, floods, earthquakes, and human digging.
The ancient Greeks and Romans of course also stumbled into these bones every now and then and were without exception baffled. Philostratus (c. 212-13 AD, On Heroes 8.3–14) gives a realistic account of one such encounter after an earthquake exposed a giant skeleton on his native island of Lemnos: "The backbone was in pieces, and the ribs were wrenched away from the vertebrae. As I examined the entire skeleton and the individual bones, I had an impression of terrifying size, impossible to describe..." Until Alexander's campaigns in India (300s BC), the largest land animal known to the Greeks was the horse or ox. In contrast, the largest fossil tusks (of a Miocene mastodon) ever found in Greece were excavated by Aristotle University team in 2009; about 5 meters long. It appears that when the earliest Greeks tried to rationalise what in the world could have such massive bones, their conclusion was: mythical heroes, giants, and monsters.
The Father of History Herodotus (560 BC; Histories 1.67-8) might have the earliest known mention of megafauna fossils. He writes that the Delphic Oracle told the Spartans to go to Arcadia in Tegea and look for the bones of the hero Orestes. After looking for a while in vain, they met a local blacksmith who had come across a skeleton in a coffin that was 12 feet long while digging a well. The Spartans reasoned that this could be nothing else than the remains of Orestes, dug up the bones, and took them to Sparta. Herodotus thinks that because of this relic, the Spartans never lost another battle against the Tegeans. There's obviously some fantastical elements here - but Tegea actually has very rich fossil deposits so it might be based on a true historical event.
And, it seems that many Greek city states kept ancient massive bones in temples and treasuries, believing them to be bones of past heroes or monsters and thus having supernatural qualities. Quite a few ancient archaeological sites have yielded paleontological treasures as well (a full list is in Mayor, 2011: 167–9). Also, the geographer Pausanias (2nd century AD), who wrote a sort of tourist guide to Greece, mentions massive bone relics a number times (e.g. 1.35.3, 5-6; 8.29.1–4; 8.46.1). According to him, the magnificent three-foot-long tusks of the mythical monster Calydonian Boar (boar was the only tusked animal the archaic and classical Greeks knew) were stored in a temple in Tegea. In Lydia, Pausanias viewed a massive skeleton, exposed by heavy rains at the river Hyllos. Locals identified the body as Geryon, the monstrous giant slain by Heracles, because Gerson famously raised colossal oxen - the bones looked like big horns. Near Troy, some bones of a 'hero' also surfaced, and this "hero’s kneecaps were the size of a boy's discus in the pentathlon". The emperor Hadrian among others came to admire these bones, and believing them to be the bones of the hero Ajax, he reburied them in a fine tomb. Pausanias also identifies the area around Megalopolis in the Peloponnese as the ancient battleground of the Gigantomachy, the mythical battle between titans and gods, because the ground there was apparently littered with gigantic bones in Pausanias' time.
I could go on forever relating more stories about ancient encounters with dinosaur finds because they are just pure fun! Every now and then the ancient Greeks and Romans did make a bit better rationalisations about the origins of the giant bones than titans, although they never reached a level of sophistication comparable to the theory of evolution or anything. Pliny the Elder (Natural History, e.g. 7.73–5, 9.10–11, 28.34, 2.221) shows some scientific interest and discusses animal, plant, and marine fossils used and collected around the Mediterranean. According to Suetonius, Augustus also made a point of collecting fossils to his villa in Capri (Life of Augustus, 72.3):
His own villas, which were modest enough, he decorated not so much with handsome statues and pictures as with terraces, groves, and objects noteworthy for their antiquity and rarity; for example, at Capri the monstrous bones of huge sea monsters and wild beasts, called the "bones of the giants," and the weapons of the heroes.
Mayor (2014: 583-4) has dug up couple of real interesting references from more obscure sources: "In about 200 BC, Euphorion (fr. in Aelian, Characteristics of Animals 16.39) told of enormous primeval beasts called Neades, which populated Samos before humans came to the island. Their huge stone bones were displayed on the island to travellers in antiquity... Significantly, this early folk interpretation of the Samos fossils was revised after elephants became well-known in the Mediterranean world. In the first century AD, Plutarch (Greek Questions 56) stated that the massive petrified bones of Samos were recognized as those of elephants, perhaps brought from India by the god Dionysus in the era of myth.. Phlegon of Tralles, a writer and a freedman of emperor Hadrian, described discoveries of bones of immense size in Messenia, Rhodes, Athens, Dalmatia, Pontus, Carthage, and Egypt. Phlegon also recounts how the emperor Tiberius had instructed a sculptor to create a life-sized bust of a giant based on immense molar from a skeleton exposed by an earthquake near the Black Sea."
One last point of curiosity: Mayor also has argued that the Greeks even left us the 'earliest known depiction of a vertebrate fossil'. The so-called Hesione Vase (c. 570 BC - 550 BC) from Corinth shows Heracles saving Hesione from the Monster of Troy, which, she thinks, is a drawing of a dinosaur skull: the articulated jawbone full of jagged teeth, the extended back of the skull, and the hollow eye socket are 'naturalistic details' of a fossilized skull. Well, everyone can make their own conclusion about whether the vase's Monster of Troy must be inspired by dinosaur skulls or not, but it's a fun theory nevertheless.
There's only one historian who's done a lot of work on this topic as far as I know, Adrienne Mayor, whom I have here copiously cited and paraphrased; here's her work that I've read.
Mayor, A. (2000), ‘The “Monster of Troy Vase”: The Earliest Artistic Record of a Vertebrate Fossil Discovery?’ Oxford Journal of Archaeology 19, 57–63.
—— (2011), The First Fossil Hunters: Dinosaurs, Mammoths, and Myth in Greek and Roman Times. Princeton.
-- (2014), "Ancient Fossil Discoveries and Interpretations", in The Oxford Handbook of Animals in Classical Thought and Life, edited by Gordon Lindsay Campbell.