David Hume, Adam Smith, James Watt, Alexander Graham Bell. All Scots who had a huge impact on the world. Nowadays Scotland is more known for things like Trainspotting and deep-fried Mars bars than intellectual and technological advancements.
What was it about the Scotland of the 18th and 19th centuries that made it such a hub of innovation? And why didn't the grand intellectual traditions continue?
I LOVE this kind of question because the pattern you are noticing is an actual phenomenon that you don't have the vocabulary to describe and Scottish history just makes me giddy. I will narrow down your scope just a tad, to the "Long 18th Century" (1688-1815) instead of including the remainder of the 19th century.
Two words: Scottish Enlightenment. Let's break it down into a few different parts:
THE EDUCATION OF SCOTLAND
The beginnings of the trends that you're picking up on in the 18th and 19th century started in the 17th century so it's imperative to look back a bit. Scottish Parliament and the Privy Council of Scotland passed a series of acts throughout most of the century (1615-1695) to financially bolster and continue to flesh out a vast network of parish-based schools that were initially established in 1496 with the Education Act (which mandated that the sons of all barons and freeholders should attend grammar schools). By the beginning of the 18th Century, the lowlands of Scotland were covered in schools. Its actively debated by historians, but there is most likely truth to the fact that young Scottish men during this time had a higher literacy rate than their contemporaries. For reference, when England and Scotland united in 1707 Scotland was about 1/5 the size of England in population but Scotland had four long established universities (Aberdeen, Edinburgh, Glasgow, and St. Andrews) to England's two (Cambridge and Oxford). When the Restoration finally rolled around in 1660 after a long period of bloody civil wars and the Commonwealth, these universities began utilizing a lecture-based curriculum that provided one of the highest quality liberal educations in the world at the time. Each university established a mathematics department. St. Andrews and Aberdeen built observatories. The Royal College of Physicians of Edinburgh was established in 1681 by the Professor of Medicine at Edinburgh which pushed the Scotland to the forefront of medical education. Students at the Scottish universities were more diverse thanks to the lower attendance cost. All of these things set Scotland up for a knockout educational system in the 18th century to begin with, but it really starts to take shape when you consider ...
THE ECONOMY OF SCOTLAND
Following the unification of England and Scotland there was a conscious effort made to bridge the gap between the two upper classes. When the American colonies really stepped onto the scene as an exporter of goods, Scotland began to make enough capital to focus attention on infrastructure and education revitalization. Glasgow became a pivotal step in the tobacco trade as a go between for the colonies and France and the Bank of Scotland and Royal Bank of Scotland were established in 1695 and 1727 respectively. With more money, comes a higher demand for the best education possible and the merchants who became so wealthy from the new trade routes invested in other areas of life such as textiles, leather, soap, breweries, etc. 1723 saw the founding of the "Society of Improvers" and the beginning of the Scottish Agricultural Revolution starts about 20 years later, which fleshes out the economy of the rural areas of the country.
THE LITERATURE OF SCOTLAND
I've briefly discussed the importance of Scottish writing during the period of the American Revolution in a recent answer for "Can the American Revolution be traced back to Culloden" but its importance was WELL established by 1776. Edinburgh alone had six individual printing houses and three paper mills by the year 1763. Only twenty years later that number grew to 16 printing houses and 12 paper mills. Scottish intellectualism revolved around the printed word. Reading societies, libraries, and clubs began popping up all around Scotland as hubs for liberal elites, artists, and philosophers to battle it out over politics, literature, and art; all of these institutions being heavily intertwined with the universities of Scotland. Scottish writers and poets dominated the international literature scene. The freaking Encyclopedia Britannica was published in Scotland for the first time in 1768 and continued to be published in Edinburgh for the next century.
Scotland's trade economy, education system, and love of printing presses created the perfect storm for an intellectual movement that would literally change the world. The Scottish Enlightenment touched everything: literature, architecture, art, philosophy, mathematics, science, medicine, and the figures of the Enlightenment went on influence the movement in Europe and America. You couldn't throw a damn stone in the western world today without hitting something that can be related to the Scottish Enlightenment in some way. A (now deleted) comment below suggested that perhaps you weren't looking wide enough or that your sample was biased if you thought that there were "so many" Scottish pioneers at the time. But it's true, there were. And they vastly outnumbered any other nationality.
As to why these grand intellectual traditions ended...that's harder to pin down but I can discuss a few aspects of it that might answer some of it. The movement of the Enlightenment began to slow around 1815 but a strong tradition of Scottish literacy and writers/philosophers continued will into the first half of the 19th century. We're going to have to get into the weeds a bit so stick with me.
The East India Company was established literally at the beginning of the 17th century. December 31, 1600. They operated as a private company and largely unregulated for most of their existence, up until the last third of the 18th century when financial difficulty, famine in the Bengal region, and an overall decrease in trade consumption hit England. Out of these things, the Tea Act of 1773 (yes, that one), and the Regulating Act of 1773 were born. A series of reiterations continued through the 19th century until the Indian Rebellion of 1857 which prompted the British government to nationalize the EIC and establish the Indian Civil Service as basically a proxy government that ruled nearly 300 million people between 1858 and 1874 when the EIC came to an end. The ICS became one of the most prestigious posts a British citizen could hold and the interest prompted an entrance exam starting in 1853 based in science and mathematics that was notoriously difficult. The liberal focus of the Scottish university system didn't prepare Scottish students well enough for such an entrance exam and when the 1858 entrance exam rolled around Scottish applicants performed noticeably worse than English applicants. Prominent Scottish figures in Edinburgh newspapers expressed the need for Scottish educational reform to better prepare young men for employment with the ICS in this new changing world and a heated debate about the pros and cons of the English university system vs. the Scottish university system was ignited. The 1858 Universities Act effectively ended the debate and put an end to the Scottish university system as it was know. This legislation was controversial and very far reaching, bringing literally medieval traditions to an end and putting philology and mathematics in a privileged position that they had never before seen in Scotland. Goodbye, lecture-based liberal education. And goodbye, world class philosophers, writers, and artists pouring out of Scotland.
Further Reading & Sources:
How the Scots Invented the Modern World, Arthur Herman (A MUST READ for anyone interested in history)
"Scottish Literacy and the Scottish Identity: Illiteracy & Society in Scotland & Northern England, 1600–1800", RA Houston
"The Child Writer: Graphic Literacy and the Scottish Educational System, 1700–1820", Matthew Eddy
"The History of Scottish Education", RA Houston
"Reading the Scottish Enlightenment: Books and Their Readers in Provincial Scotland, 1750–1820", Mark Towsey
The Transformation of Scotland: The Economy Since 1700, Devine, Lee, & Peden
The Democratic Intellect, George Elder Davie
We first must consider if it is more likely that Scottish intellectual achievement has fallen since the 19th century? Or that that global intellectual achievement has risen.
The innovators you have listed are all tied to the industrial revolution. So let us consider that in its geopolitical context.
https://commons.wikimedia.org/wiki/File:Europe_1815_map_en.png France is still reeling from the French Revolution, and the Napoleonic wars have ravaged most of Europe. Overrunning Spain, and the countries now present in modern day Italy. Most Russian cities were burnt down, largely by the Russians themselves in their war of attrition against Napoleon. Though Prussia did not suffer a similar fate, they had been committing the bulk of their resources into war preparation and fighting Napoleon. The Ottomans were not involved in the direct fighting, but their Empire was well on its decline as of the 1800s due to internal corruption. And the non-European powers of the world, were either colonies or decades away from the capacity for an industrial revolution.
The United Kingdom was the only major power to come out of the Napoleonic era relatively unscathed. Owing to the superior British Navy, French troops never set foot on English soil. Couple this with the British Empire, which now spanned a quarter of the world's landmass, and you have a recipe for economic prosperity like none-other.
There are 4 principal factors that lead to the UK's industrial revolution. Availability of Capital, Sociopolitical Stability, Enormous Overseas Trade, and a Small Population. Capital is necessary to invest in innovations, and stability is necessary for inventors to devote themselves to their work. Thanks to the success of the British in the Napoleonic Wars, the UK had both in spades. The next two factors go hand in hand. The population of the British Isles was unable to keep pace with its ever-growing trade. Thus, this necessitated the creation of new devices to keep production in line with the growing demand.
This is best exemplified by the changes in the textile industry as well as the coal industry. The shortage of labor force compelled industrialists to encourage and apply new mechanical devices. Watt's steam engine was intended to pump water out of mines.
Keep in mind. It is perfectly possible that other men in other parts of the world had the same ideas as the innovators of the UK. But lacked the same socio-economic conditions that allowed UK innovators to be successful in their endeavors.
If you've stayed with me this entire time, you must be wondering. Why Scotland? The UK of the 19th Century encompassed the Irish, the Welsh, and the English. So why were the Scots the only inventors? The simple answer is that they were not.
The list of English inventors who contributed to the industrial revolution is equally impressive, and significantly longer. Such as Jethro Tull who invented the Seed Drill, Thomas Newcomen who devised the first practical steam engine, Edward Jenner who invented the first Vaccine, Abraham Darby III who built the first Iron arch bridge, and Charles Babbage who created the Difference engine, an automatic mechanical calculator designed to tabulate polynomial functions. The difference engine is especially fascinating because it is the intellectual ancestor of all modern computers.
In all honesty? Nothing happened to Scotland. The University of St Andrews, the University of Glasgow, and the University of Edinburgh are still regarded as some of the best in the world. They don't have any shortage of Nobel Prize winners or major intellectuals.
The Scots never declined, the rest of the world caught up. During the Industrial Revolution in the UK, there were no competing innovators from India, China, or South Africa. The universities that lead Silicon Valley, Berkeley, and Stanford, had only just been founded.
There were many pioneers of Industrial philosophy and technology in Scotland because of the social-economical-political factors present in the UK. The grand intellectual traditions of Scotland have continued, but no longer look as impressive when compared with the rest of the world.
I addressed what I felt was the core of your question. If you want the nitty-gritty of the exact conditions in Scotland. Things like the Political Economy Club, where Adam Smith first debated and discussed with merchants and academics. Then I would suggest you pick up a book on the Scottish Enlightenment. I recommend The Scottish Enlightenment by Alexander Broadie. https://www.amazon.com/Scottish-Enlightenment-Alexander-Broadie/dp/1841586404
Follow-on question: in what ways was this fuelled/funded/enabled by slavery?
