Yes, White is generally believed to have been a Soviet agent. US counterintelligence had suspicions about his loyalties from the early 1940s, but didn't reach a conclusive verdict until after his death in 1948. It's only relatively recently that documentary evidence has been made public which appears to conclusively identify him as having passed information to the USSR.

He appears to have provided information to the USSR prior to the Second World War, and stopped around 1938 before being reactivated in the early 1940s. (That late 1930s lapse in activity correlates with a generalised decline in the activity and efficiency Soviet intelligence operations around that time, thanks to Stalin's repeated and sustained purges which threw the NKVD and military establishments into disarray.)

As with any spy, there are a range explanations for why he chose to pass information to the Russians, but there really doesn't appear to be any doubt left that he did work with Soviet intelligence, and that he did materially support the USSR from his position within the Treasury department.

Here's a 1950 memo sent to FBI director J. Edgar Hoover by assistant director D. Milton Ladd. He reports that Venona decrypts had allowed US counterintelligence to identify White as a Soviet agent codenamed JURIST:

On the basis of the foregoing, the tentative identification of Harry Dexter White as Jurist appears to be conclusively established inasmuch as Morgenthau and White left the United States on a confidential trip to the Normandy beachhead on August 5, 1944, and they returned to the United States on August 17, 1944.

Vasili Mitrokhin, a KGB archivist who defected to the UK with a vast trove of First Chief Directorate (the KGB's foreign intelligence division) documents in 1992, also identifies White as having been a Soviet agent. From The Sword and the Shield by Mitrokhin and Christopher Andrew:

Even in the mid-1930s the main Soviet espionage networks in the United States were run by the Fourth Department (Military Intelligence, later renamed the GRU) rather than by the NKVD. Fourth Department agents included a series of young, idealistic high-flyers within the federal government, among them: Alger Hiss and Julian Wadleigh, both of whom entered the State Department in 1936; Harry Dexter White of the Treasury Department; and George Silverman, a government statistician who probably recruited White. [Chapter 7 — referring to an ebook version, so can't give a more accurate page reference.]

From KGB: The Inside Story of its Foreign Operations from Lenin to Gorbachev by Andrew and Oleg Gordievsky, on White's role in post-war diplomacy and economic policy, and his covert and overt activism in support of the USSR:

In the economic negotiations at Yalta, centered on the issue of reparations, Soviet negotiators were powerfully assisted by Harry Dexter White, the most important of several NKVD agents in the U.S. Treasury. Since 1942, as the closest adviser of Morgenthau, secretary of the Treasury, White had taken the leading part in formulating American policy for the international financial order of the postwar world. Together with Lord Keynes, he was the dominating figure at the Bretton-Woods Conference of July 1944, which drew up the blueprint for both the International Monetary Fund and the International Bank for Reconstruction and Development.

[...]

Though not present at Yalta, White had already given strong backing to the Soviet case. In January 1945 he took the lead in drafting two memoranda sent by Morgenthau to the president. The first proposed a thirty-five-year loan to the U.S.S.R. of $10 billion at 2 percent interest.

[...]

White's advocacy failed to overcome State Department opposition to the $10 billion loan to Russia and to the dismantling of German industry. [...] White, however, had already facilitated a concealed American subsidy to the Soviet Union. In 1944 White provided the NKVD through Silvermaster with samples of occupation currency printed by the Treasury for use in German. Thus prompted, the Russians decided to ask for the plates, ink, and paper samples in order to print notes of their own. The director of the Bureau of Printing and Engraving reasonably objected that "to permit the Russian government to print a currency identical to that being printed in this country would make accountability impossible." White protested that the Russians would interpret this as showing lack of confidence in their integrity; they "must be trusted to the same degree and to the same extent as the other allies." A week later they received the plates. In 1953 a Senate hearing was told that "there is no way of determining just to what extent the Russians did use these plates." The cost to the American taxpayer may well have run into millions of dollars. [Andrew & Gordievsky, KGB (1991), p335-337]