The men who wrote Yes Minister (Anthony Jay and Jonathan Lynn) did so using information supplied to them from real government officials, and in using it tried their best to make a show about bureaucracy funny. But to what extent did the Civil Service really subvert its political masters during the last century’s latter half? It’s hard to answer such a broad question and when you’re dealing with an organization that encompasses the entirety of a G7 government, you’re bound to get a lot of variance in experience. Practically, I don’t think I can give you an answer that covers the entirety of the second half of the 20th century, so let’s instead look at the views and experiences of the people who influenced Yes Minister and what they can tell us about the realities of the Minister-Whitehall relationship.

First, some context for anyone unfamiliar with Yes Minister and the system it satirizes. The UK government is divided into departments for different policy areas, and each department is headed up by a Minister (often called a “Secretary of State” for that area or just “Secretary”). So foreign affairs are overseen by the Foreign Secretary, crime and immigration has the Home Secretary, health has the Health Secretary etc. These people are Members of Parliament (MPs) elected to represent constituencies around the country in the House of Commons, Parliament’s lower (and dominant) chamber. In UK elections, candidates from the various parties run in each constituency and (almost always) the party that elects the most MPs (usually an overall majority) forms the government. Ministers are named to their departments by the Prime Minister, who is almost always their party’s leader. Members of Parliament’s upper chamber (the House of Lords) can also be made ministers but this is usually for more junior posts.

A department’s work is carried out by civil servants, apolitical officials who implement the government’s agenda. The civil servants in each department are organized into a hierarchy of roles and responsibilities headed up by someone called a Permanent Secretary. A Minister works with a Permanent Secretary (Perm Sec) to run the department, with the Minister accountable to Parliament for what the department does. Ideally, the democratically elected government implements its policies with the advice of an experienced and apolitical Civil Service, which makes the government’s wishes into reality.

In Yes Minister, an MP called Jim Hacker runs the fictional Department for Administrative Affairs (DAA), whose Perm Sec is called Sir Humphrey Appleby. The theme of the show is Jim Hacker trying to accomplish governmental goals, and Sir Humphrey impeding him with bureaucratic obstructionism. In Yes Minister, the Civil Service is a never-changing government run by privately educated Oxbridge elitists obsessed with maintaining the status quo. What’s more, the Sir Humphrey model of civil servant is a generalist with no specific expertise in the operations he oversees. He did Classics at university, not some obscure technical discipline. This generalist class of administrators is eternal in Yes Minister’s world. Politicians come and go, but the Civil Service stays the same.

Now to get into some actual history (finally).

Richard Crossman was a Labour MP first elected in 1945. In 1964, he became the Minister for Housing and Local Government under Prime Minister Harold Wilson, and held a couple more portfolios after that. Before he was a Minister, he criticized the post war Labour government of Clement Atlee for its “uncritical reliance on Whitehall.” He thought that a good left-wing government should rely more on expert advice “from an influx of experts with special knowledge, new ideas and a sympathy for the government’s domestic and foreign policies.”

He brought his views on the Civil Service with him when he became a Minister. In 1966, he told a special committee that was examining Civil Service reform that “the higher Civil Service was a coherent and cohesive oligarchy presenting Ministers with narrow alternatives of choice.” Crossman’s diaries – published posthumously in 1975 – recounted his rows with the Perm Sec at Housing, the formidable Dame Evelyn Sharp. The diary’s publication came in spite of concerted opposition from the government since – according to Lynn – “the diaries revealed that the Civil Service runs the country.” So, from that perspective you could say that (for 20 years at least) the Civil Service had been subverting Ministers to quite a considerable extent.

Since Lynn calls Crossman (through his diaries) “our best source,” it’s worth looking at some examples of his relationship with Whitehall. When Crossman was Housing Minister, Labour wanted to create a new Ministry of Land and Planning to manage the supply of land for the implementation of its agenda on housing. This meant transferring land use planning authority from Sharp’s ministry. Crossman was supportive of the plan but Sharp managed to keep the relevant powers under her department’s purview. In 1969, Crossman (now at Health and Social Security) had to overcome Civil Service resistance to creating the Hospital Advisory Service, which reported on conditions in hospitals for the elderly and mentally ill. There he was successful, but he had to work for it. In the same year, he asked his officials to look into reallocating 11% of general hospital expenditures toward targeted spending on long-stay hospitals with a priority on the mentally ill. This generated intense opposition from the health bureaucracy and Crossman had to abandon the idea, instead settling for much less money from central department reserves. There’s a lot more to Crossman’s work on this issue but I’m already running long so I’ll leave it there (I’m already probably getting too big for my britches, as I’m by no means an authority on UK healthcare or housing policy).

That’s just 3 (very simplified) examples from Crossman’s 6 years in government, but they paint a picture that’s eerily reminiscent of Yes Minister. Still, they’re hardly indicative of his overall experience, and while his relationship with the Civil Service was strained, he came to understand its inner workings and how to make it work toward his goals. In 1971 he delivered a lecture where he said

My final lesson is this. People often say to me “Look Mr. Crossman, the system. It really doesn’t work does it?” Let me tell you something. I spent six years inside the system. I think it works extremely well…We have a Civil Service which will carry out the programmes of politicians, provided the politicians are willing to fight day and night for their ideas. Of course if you will not fight for your ideas, if you simply say to the civil servants “Please carry this out,” they will not. But once you know you are serious a change takes place. Suddenly it is all sweetness and light and you can get it done. I see nothing in the system which would have stopped us from doing much better than we did. If we failed, it was because we made mistakes. I think I would conclude by saying, “The fault, dear Brutus, lies not in our civil servants but in ourselves.”

I don’t think that’s the view of a man who was stymied by the Civil Service at every turn. It strikes me more of a pragmatist who learned how to understand a complex system and how best to work within it.

Moderated though they may have been with time, Crossman’s views on the Civil Service were hardly unique. Harold Wilson’s advisor Marcia Williams (a source for Lynn and Jay) felt that Labour’s failings in government were attributable to the work of a Civil Service she mistrusted and disliked. The same was true for Wilson’s economic advisor Thomas Balogh, who railed against what he called an “ignorantly dilettante bureaucracy.” He felt that civil servants kept information from Ministers for their own gain, and complained to Wilson that “a large part of the bureaucracy does not share our views.” When Wilson returned to office in 1974, he appointed Bernard Donoughue to set up the new Number 10 Policy Unit, which would have to work closely with civil servants. Donoughue ended up being an important source for Lynn and Jay for both plot ideas and checking scripts. He had a complicated relationship with Whitehall, especially the Treasury, one of the most powerful departments, which was being potentially undercut by a new source of independent economic advice within Number 10. Still, a working relationship developed that was antithetical to the anti-Civil Service attitudes of Balogh and Williams. While they were both sources for Lynn and Jay, they always met Donoughue and Williams separately, as the two hated each other.

The upshot of this answer to a question you didn’t really ask is that the Civil Service depicted in Yes Minister is one informed by the views of people who had complicated relationships with it. In the officials described in Crossman’s diaries, as well as the recollections of Williams and Donoughue, we can certainly see the bureaucratic obstructionism of Sir Humphrey and the capacity to subvert the will of elected representatives. Still, Crossman’s self-reflective words remind us that blaming every failure on obstinate bureaucrats is at best a simplistic view that shirks responsibility.

If I had more time and energy, I’d love to get into a discussion of the Civil Service pre-1964, its relationship with Heath, and its evolution under Thatcher. Maybe someone else can fill in some blanks there, and while they’re at it poke holes in what I’ve written (I’m sure there are plenty). Still, I hope that this has provided some insight into the (Prime) Minister – Whitehall relationship, and how to view the bureaucracy's famous tendencies to thwart the will of its political masters.

Sources and further reading to be included in reply to this comment.