You’ve asked a big question. I’m going to tackle this on a couple of different lines.

First, remedially: authorial intent is still very much alive in English scholastics. The footnotes of many essays across the field are full of references to author’s letters, drafts of the works being studied, diary notes ⁠— the author, for many English scholars, is very alive. The study of English as we understand it today is a young discipline, and there’s still lots to work out. Like anything in academia, you can still get a pretty good fight going over it.

First, I think we should talk about what the essay “La mort de l'auteur” (Barthes’s little pun on Mallory’s “La mort d’Arthur”) is and isn’t. Brian Dillon, in his lovely book Suppose a Sentence ⁠— it’s really a joy, you should read it ⁠— offers a good way (not the authoritative way) to think about Barthes. He says, essentially, that Barthes’s trick is that he’s offering a view of a fantasy world, a world in which the national character of Japan can be understood from a perfect piece of sushi, and all human sexuality can be seen in a poster of Brigitte Bardot. But that’s all it is: a fantasy. Barthes is taking us on a journey to a fantasy land, and shouldn’t be read perfectly literally ⁠— but the existence of the fantasy can still give us much to reflect on in the real world.

In “The Death of the Author,” Barthes, I believe, does it again. He offers a fantasy, that the author, which he calls “the scriptor,” isn’t really writing, but is instead sort of shamanically channeling all of life into the work, and all the work is being channeled into the author, until, like the worm Ourborous, “life can only imitate the book, and the book itself is only a tissue of signs, a lost, infinitely remote imitation.” This is the part that usually gets glossed over, because Barthes at his weirdest is a very odd man, and nobody who teaches English below a graduate level wants to get into this tangled and confusing fantasy-world ontology.

So, the part that winds up being very influential is the introduction and parts of the main text that don’t have to deal with this literary Ouroboros. For space, I won’t quote the passages in full, but they’re very good, and very worth reading. He argues, essentially, that it’s impossible for anybody, even the author, to piece together what, exactly, they meant, because there are too many events and influences ⁠— the author’s past, their subconscious, the state of the world, the local news, what they last read, etc. ⁠— to untangle into discrete threads. “All writing is itself this special voice,” he says, “consisting of several indiscernible voices.” As such, because it’s impossible to mine even our own intentions, much less the intentions of strangers, we may as well just leave intent off the table and focus on the text at hand.

That is, in a nutshell, what the death of the author is. Where it came from is another thing entirely.

For the following paragraphs, I’m indebted to Barbara Herrnstein Smith’s essay “What Was ‘Close Reading’?: A Century of Method in Literary Studies.” I’m also drawing off TS Eliot’s The Sacred Wood: Essays on Poetry and Criticism

The essay “Death of the Author,” was written in 1967, and, in my mind, owes its existence to the 1920s and the “New Critics.” The New Critics were a group of essayists ⁠— TS Eliot was one of them ⁠— who advocated a new way of thinking about texts. Prior to this moment in history, the major form of literary criticism that we would recognize as criticism was philology, a branch of English study, now-dead, that emphasized the study of the text’s linguistic etymology and the regional history of where the author lived (Tolkien taught philology, and it’s very clear, when you read LOTR, that he did).

Besides philology, Smith says, the English scholar pre-1920 would have been concerned with “the production, transmission, and acquisition of facts about sets of texts” ⁠— names, dates, and types of work assembled into compendiums.

The New Critics emerge on the scene and completely change the game. They see texts as living objects to be interacted with, to be pulled on and looked at to understand what made the poem or novel tick. They were like watchmakers looking at mechanisms under loupes, but, like watchmakers, they weren’t really interested in what the watch meant ⁠— they just wanted to see how the watch worked. To do that, they used a common analytical tool called “close reading”.

Close reading is a way of engaging that, at its most basic, involves taking a piece of text and working with it exclusively ⁠— you put the poem in isolation and figure it out from there. You forget the author’s name and anything else, you just look at the words, the commas, the stresses, and come up with an analysis of the piece. Now, that’s just a very basic way to do a close-reading, and there are many techniques and rhetorical moves that you can do with them that involve bringing outside sources in, but for the moment, that’s a working definition. This is almost a proto-death of the author: for a moment, you and the New Critics plunge yourself into pure text, ignoring everything outside of it, coming back for air when you need to.

Close-reading itself has a long history in the world, because arguing about verbs and commas is a time-tested hobby. The rabbis in the temple of Nazareth that Jesus quarreled with would have been very familiar with the ideas of, and possibly some techniques of, close-reading, if not the vocabulary.

Now, the New Critics would have rejected Barthes. For example, Eliot was a great proponent of the importance of understanding all the author had done to that point before drawing a judgement on the work. They did not reject the idea of authorial intent. However, the New Critics weren’t scholars in the modern sense ⁠— they were mostly literary critics trying to pass judgment on works, whether or not the art worked. But their methods and ideas get picked up by scholars and students, who love this new and engaging way of working with texts. This had to be a very exciting moment ⁠— you go from cataloguing all of Keats’s influences and sticking the resulting encyclopedia on a shelf to actually working with the poems themselves, trying to grasp them and understand them in a really complex and nuanced way

Sadly, I don’t know why close-reading becomes the major analytical tool of the 1940s and ‘50s. I’ve tried to find out before, but I haven’t gotten a good answer. The best I’ve heard was from an English professor, who said that it was a fantastic pedagogical tool for teaching veterans on the GI Bill. If you’re trying to teach Shakespeare, but the class doesn’t know any Ovid, because they grew up on farms and spent their youths reading Dick Tracy, close-reading is an invaluable tool, because it allows you to set that all aside. I have doubts about this theory, but I don’t know enough to say that he was wrong.