As the Hollywood Studio System gained steam in the 1920s, the idea coalesced that the Producer was the prime creative force in a film. When the Academy Awards was started by the Academy of Motion Picture Arts and Sciences, there's a reason that there was no individual award for Producing. The director was considered as much of a craftsman as the Cinematographer, Screenwriter, or Editor, and given their own award, while the Best Picture award went to the Producer.

This was galvanized during the 1930s, when people paid the most attention to the Studio that released the film in judging likely tone and quality. Production units of recurring creative personnel coalesced around the Producer rather than the Director.

There were of course some outliers to this, most notably Alfred Hitchcock, who was one of the first major directors to bend powerful producers and studios towards his vision of a film. A major example is his first American studio film, Rebecca, which was produced by David O. Selznick, who like all Producers of his day received the final edit of his films. Hitchcock very famously spited Selznick by editing the film in-camera. That is, he had edited the film in his head before he shot it, and would only film the angles he knew would be in the final film, thus making it impossible to coherently edit the scene in any other way.

The break from studio-producer dominated creative teams started in Europe, as many of the postwar films made there were done without the support of major studios. The Italian Neorealists like Roberto Rossellini and Vittorio De Sica stand out particularly here.

At the same time, the Supreme Court decision of United States v. Paramount Pictures, Inc. in 1948 was a landmark decision in the history of cinema. The Studios were forced to divest themselves of their monopoly on movie theater ownership. As a result, theaters were now able to widely book foreign and independent movies.

Over the course of the 1950s, the prominence of the Producer began to fade. There's an illustrative anecdote that immediately springs to mind. Singin' in the Rain was one of the biggest blockbusters in 1952, indeed in the entire golden age of musicals in the early 50s. MGM was then the biggest studio, and it had three Producers: Arthur Freed, Joe Pasternak, and Jack Cummings. Freed was the most well known of these, and he produced the biggest musicals with the biggest budgets, and he produced Singin', which was co-directed by Stanley Donen and star-cum-choreographer Gene Kelly. Two years later, Kelly made another movie with Freed, the high budget Brigadoon, while Donen was transferred to Cummings's lower budget unit, and made Seven Brides for Seven Brothers. Well when Brigadoon flopped and Seven Brides for Seven Brothers was one of MGM's most wildly successful movies of the year, Donen was wildly heralded as a talent to watch out for. He had been one of the greatest directors for both Kelly and Freed, and demonstrated his indispensable talent elsewhere.

The watershed moment came in the same year. Cahiers du cinéma was a French magazine founded in 1951, and in 1954, film critic and future director François Truffaut wrote an essay in which he argued that the Director was the auteur, or author, of a film. The Director, he argued, had more creative control than even a screenwriter, because the director, responsible for the staging, framing, and editing of a film, could dramatically change the tone beyond the written word. As a side note, Truffaut and Jean-Luc Godard had a lunch in Paris with the aforementioned Roberto Rossellini shortly thereafter, and Rossellini basically dared them to get off their asses and start making movies themselves. French New Wave was born shortly thereafter.

Meanwhile, Hollywood was discovering how to use the Director to market their films. Alfred Hitchcock, again, was at the forefront of this, having become essentially the first celebrity director, someone that people could recognize on sight. Look at his movies in the late 50s: he consistently used the same editor, composer, cinematographer, and other key creative personnel on most of his projects during that time. In addition, the Producer and Screenwriters answered to him.

The auteur theory came to America when Andrew Sarris started writing about it in 1962, examining the creative output of Hollywood not from studios or producers or stars, but by analyzing consistencies in the output of the Directors. By 1963, the power of the studios had mostly been crushed, and independent production that vested much more creative control in the now hyped position of Director.

Significant talents, such as Kubrick really took off in the mid to late 60s, and you can see as the years progressed, they began to often take Producer credits on their films. When the New Hollywood era launched in the early 70s, criticism had placed key creativity in the directors, and the business model had adapted to use the director to market films.

Source: I received my MA from the USC School of Cinematic Arts, you can find sources for all of this in: The Oxford History of World Cinema, The Golden Age of Cinema by Rick Jewell is highly recommended, The Classical Hollywood Cinema by Bordwell et al, The Hollywood Studio System by Gomery, Hitchcock/Truffault, Can't Help Singin' by Mast, A Certain Tendency of the Hollywood Cinema by Ray, Postwar Hollywood by Casper