What do historians think about the sort of work done by economists in this paper?

by ghostof_IamBeepBeep2

From the National Bureau of Economic Research: History’s Masters: The Effect of European Monarchs on State Performance

The pdf is here. I guess I'm curious about both this text and texts like it more generally.

In the abstract, they note:

We code the degree of blood relationship between the parents of rulers. The ‘coefficient of inbreeding’ is a strong predictor of ruler ability, and the corresponding instrumental variable results imply that ruler ability had a sizeable bearing on the performance of states and their borders.

What do you or others think about this sort of quantitative methodology? Is it the sort of thing historians would be familiar with?

And finally, are the findings in this paper (or papers like it) accurate? How are they received by historians?

IconicJester

I don't know if you're going to get much response about the paper itself, which is a brand new working paper that has not had any time at all to circulate even within economic history, let alone more broadly. Perhaps someone has seen it presented at a conference or seminar, though I haven't.

I can tell you that Nico Voightlander is well-respected as an economic historian, though coming from the quantitative and econ-focused side of the discipline. His work with Joachim Voth on historical demography and growth, on the persistence of anti-Semitism, and on the rise of the Nazis has all been well-received. His work fits into the broader pattern of empirical economic history from the econ side, making very large claims on the basis of powerful methods, but which are contingent on the quality of the data and the precise specification of their model. This is very far indeed from a sort of (simplified) von Ranke "read the primary sources and tell us what happened" method for doing history. Sebastian Ottinger I do not know, but a quick google suggests he is a job market candidate finishing his PhD at UCLA.

Applying econometric techniques (specifically, using instrumental variables (IV)) for distinguishing causal relationships has been de rigeur in economic history (and for economists looking to publish by applying their statistical tools to underused data) ever since 2001 and Acemoglu, Johnson and Robinson's Colonial Origins of Comparative Development. (If you want an overview of the methods, Angrist and Pishke's Mostly Harmless Econometrics is a good overview of the logic of natural experiments and instrumental variables. If you want a more general overview of quantifying and graphing causal relationships and their logic, try Judea Pearl's The Book of Why.) This is standard in economic history as practiced in econ, and also in the (few) standalone economic history departments. My impression is that it is somewhere between rare and non-existent in most history departments, especially in North America.

The basic idea is that, we observe two phenomena, X and Y - in this case, a ruler being skilled at ruling, and economic growth during their reign - and we want to know what the causal relationship of X on Y is. We want to know whether one causes the other, but are worried about the problem of endogeneity: that is, the relationship between X and Y is not a simple one-way causation, but rather they cause each other, or are both caused by some common thing Z. That is, we cannot rule out that the skill of a ruler is affected by economic growth. Or there could be some further factor that leads to both economic growth and good rulership. Neither of these can be conclusively ruled out from the beginning.

In an ordinary scientific setting, they would do this experimentally, using randomization and controls. But since they cannot manipulate history experimentally, they do it quasi-experimentally - they find something that causes Y, but does not affect X. In this case, they try and use primogeniture and inbreeding as instruments. They posit that these things that cause ruler skill, but cannot have been themselves caused by economic growth. That rulers neither knew the extent of inbreeding nor understood its effects at the time, it is argued, reinforces the quasi-random nature. Whether a state was primed for economic growth or not at the beginning of a ruler's reign was not a function of that ruler's inbreeding. (Whether any of this is true, I would have to defer to someone who actually understands these things; this is merely an explanation of the logic in the paper.) So armed, they can estimate the effect that ruler skill has on economic growth - the original question of interest.

These sorts of models are vulnerable to many kinds of critiques. (Which is not to say they are not useful, only that their power relies on the interaction of many moving parts.) For one, the specification of the model needs to include all the relevant causal factors, or else it will suffer from omitted variable bias - a fancy way of saying that if you've not included something causally important in your model, your results will be anywhere from slightly off to complete nonsense. Instrumental variables can go some way towards mitigating this problem, but I would be very sceptical about anyone who leans heavily on the method alone to simply ignore other relevant considerations. Without good models, you can't do good social science.

For another, your data must accurately reflect what they are supposed to represent. If they are inaccurate, they will not yield sensible results: Garbage in, garbage out. In this case, they are relying on Frederick Adams Woods' early 20th century attempts to quantify the attributes of rulers. I don't know how accurate or useful this data is, this is the first time I've heard of it. A brief description of the source sounds intriguing, but also a bit alarming; I'd have to look into it further to offer any useful analysis. The authors argue that, even if his data are biased, they should be biased against rather than towards their results, because his hypothesis is that skill should be hereditary. (This sort of "biased against us" is a frequently made argument. I find it dubious.)

If they are accurately measuring something, but that something does not match the interpretation they put on it, then the argument will be misleading. This literature is notorious for using proxies; if you can't measure something directly, you measure something else that should be closely associated with it that you can measure, and use that as an indicator. Insofar as you have a bad proxy, you have a bad argument. I don't know how much that affects their argument here, but they are certainly not measuring every part of this relationship in clear, replicable detail.

And last, you need a good instrumental variable, that closely replicates the logic of a real randomized experiment. Weak instrumental variables or ones that do not satisfy what is known as the exclusion restriction (that is, do not rule out the endogenous causality you are trying to eliminate) will yield incorrect results. If, for instance, there is more complex feedback between economic growth and inbreeding that transcends generations, such that (for instance) economic growth in one century increases the chance of inbreeding among rulers, but also decreases economic growth in the following century, then we have a bit of a problem with our instrument. Is this the case? I have no idea, but it could well be a threat to their identification strategy, in the parlance.

Do I believe the results? Depends on what you mean. Do I think skill in rulership is a real (if extremely vague) thing, and that it has an effect on economic growth? Seems plausible - at the least, incompetent rulers can make policy errors, and it's hard to believe that these have no negative growth consequences. Do I believe the argument from this paper specifically? I tend to be grumpy and sceptical about big results resting on novel IVs, and doubly so if they rest on shaky quantification of something like "ruler quality" from an early-20th century Social Darwinist. These kinds of pseudo-quantitative data can be anywhere from slightly misleading to complete junk. So I would file this under "interesting, but quite far from conclusive," and would want to read it over more carefully, and hear from both the authors and from critics before I thought more about it than that.