What are the origins of this trope, and how connected is it to the emerging fields of paleontology and archaeology of the time?

The ultimate origins of the trope go back at least to antiquity, with travelogues of strange places such as the Histories of Herodotus and The Travels of Sir John Mandeville (14th century) (and, to an extant the 1,001 Nights, although this is properly part of "Oriental" literature, the emphasis on the voyages of Sinbad and exotic locales inspired considerable later works). The 15th century began a race among European nations for exploration and colonies abroad, and the unknown potential of these lands created an impetus for fictional voyages, such as Johnathan Swift's Gulliver's Travels (1726). The 19th century saw the transformation from this exotic exploration fiction - which was often fantastical and sometimes satiric - into a genre of adventure fiction with H. Rider Haggard's King Solomon's Mines (1885).

You really have to contrast the "Lost World" against the "New World" and the "Old World" - this was a genre that was not so much about discovering an unexplored continent as much as in the uncharted parts of the map were world that had been lost - civilizations long thought dead or destroyed that survive in some strange remnant, like the subterranean Roman State of Land Under England (1935), or valleys where dinosaurs lived as in Journey to the Center of the Earth (1864) or The Lost World (1912) - places definitely inspired by both the burgeoning colonial interests of European empires, but also by archaeological discoveries and historical interest; the "Lost World" genre was an excuse to explore the living relic of ancient civilizations. Quixotically, the idea of a lost world can only really make sense in a world that is quickly becoming mapped.

"Lost World" stories are often also "Lost Race" stories - the idea being the these civilizations were so closed-off that entire peoples and nations lost to history could survive in pockets. This varied considerably from the historical to the fantastical - Edgar Rice Burroughs' Tarzan famously encounters both a lost colony of European crusaders in Tarzan, Lord of the Jungle (1927) and Romans in Tarzan and the Lost Empire (1928) in Africa for example. One of the primary characteristics of many of these "Lost Races" is that they tend to be white - echoing both legends of white peoples or tribes in sub-Saharan Africa, Asia, the Americas, etc. - and racialist ideas of vanished white civilizations, like the Hamitic hypothesis in regards to Africa. More than a few of these races, like the living dinosaurs, took on elements of fantasy - Edward Bulwer-Lytton paved the way with The Coming Race (1871), but it was adopted by both writers and occultists, and fed into the relatively contemporary idea of "lost advanced civilizations," such as Atlantis and Mu, especially as depicted by Augustus le Plongeon and Col. James Churchward - both of whom had strong archaeological interests, despite publishing what today would be considered crank theories. The "Lost World" genre both fed on real-life archaeological research and fed into such "alternative history" ideas, spawning further references in fantasy fiction, scientific, and occult circles - or example, the concept of "Shangri-La" was invented wholesale in James Hinton's Lost Horizon (1933) (although inspired by aspects of Tibetan Buddhism).

The pivot for "Lost World" fiction was Edgar Rice Burroughs, who took the general shape of the romantic adventure genre and changed the focus towards the fantastic, showing that the colonialist themes inherent in the genre worked equally well in outer space - as in Under the Moons of Mars (1912) - or in the center of the Earth, with At the Earth's Core (1914). These novels were serialized and published in the early Munsey pulp magazines, and with their many imitators ushered in the age of the "scientific romance."

A really good look at the "Lost World" novel from a literary perspective is John Reider's Colonialism and the Age of Science Fiction. For more on the influence of "Lost Race" on pulp specifically I'd suggest Sam Moskowitz' Under the Moons of Mars : a history and anthology of "the scientific romance" in the Munsey Magazine, 1912-1920.

Were they inspired by the exploits of Hiram Bingham, Howard Carter, and the like?

Sortof. Archaeology and anthropology both sparked imaginative interest among readers and writers; Heinrich Schliemann's excavations at Troy in 1873, and the meeting between Morgan Stanley and David Livingstone near Lake Tanganyika in 1871 both made headlines around the world, for example, but the fiction this inspired was not always "Lost World" or "Lost Race" fiction - sometimes it was, however fantastic or adventurous, just stories set in an exotic colonial setting, like Joseph Conrad's Heart of Darkness (1899) or Edward Lucas White's "Lukundoo" (1927). Carter's discovery of the tomb of Tutankhamen in 1922 has inspired a vast amount of literature - especially concerning mummies and curses - but relatively little "Lost World" fare. The focus was on the discovery of living marvels, which is part of the inspiration of works like Jurassic Park (1993) - which was knowingly followed by The Lost World, a node to A. Conan Doyle's classic. "Lost World" and "Lost Race" stories could and did partake of any historical period - and a few ahistorical ones, like Atlantis before it sank.

How did this influence later pulp magazine stories?

H. Rider Haggard started the "Lost World" stories as hardback novels; later stories were serialized in magazines, and Edgar Rice Burroughs' first novels of Tarzan and John Carters of Mars were serialized in the early pulp magazines of the 1910s at the same time A. Conan Doyle's The Lost World was serialized in the more upscale British magazine The Strand. The individual influence of Haggard, Burroughs, and Doyle on pulp fiction is probably incalculable. Burroughsian "scientific romance" in particular would dominate American pulp science fiction into the 1930s, having a lasting influence on writers like Robert E. Howard (whose Sword & Sorcery stories borrowed heavily from "Lost World" genre, but often transplanted the setting to a mythic past), H. P. Lovecraft (whose "The Mound" is veritably a pastiche of Burroughs' Pellucidar tales, and whose At the Mountains of Madness certainly borrows from the genre), and many more. Beyond that, many of the ideas which the "Lost World" novels originated, borrowed, or expanded upon - a technologically advanced Atlantis, "hidden" civilizations, secret peoples with fearsome powers, Shangri-La - were directly adapted into pulp fiction, and through their successors, comic books. It should come as no surprise to recognize elements of "Lost World" novels in the Wakanda of the upcoming Black Panther movie, for example, or of "Lost Race" stories in the Amazons of Wonder Woman, or the Asgardians of the Thor films. Those elements come directly from the comics, which were inspired by the pulps.

To a large degree, the "scientific romance" set the stage for later science fiction which used the conceit of exploration to examine new worlds, either in an action-romance sense (Stargate and Star Wars films) or in a satiric or philosophical sense (the Star Trek television series); they also strongly influenced the idea of "race" in science fiction in the sense of "species" - especially the humanoid races with attractive opposite-sex partners, such as Dejah Thoris of Edgar Rice Burrough's Mars novels - which has helped allow alien races to stand in so effectively for contemporary concepts of race in some works. There is rarely a "pure" "Lost World" or "Lost Race" story written and published today - the racialist ideology, anthropology, geography, and politics that went into its genesis are rarely there any more to support it - but you can still catch glimpses of it in contemporary media, especially adaptations of older material.