EDIT: Charles Koch, sorry. Type not intentional.
This seems like an improbably successful program for any president, left or right; a government that managed to tax and spend less but still achieve full employment. Did Coolidge really manage these things? Was there a catch? And how do historians generally rate the legacy of 'Silent Cal'?
Source of quote: Exclusive Interview: Charles Koch On How To Save America - Forbes
Some yes, some no.
Government Expenditures:
According to this table - warning .xls file download from whitehouse.gov, total federal outlays in 1923, the year Coolidge became president were 3.85 billion dollars. In 1929, they were 3.86 billion dollars. In the intervening years, there was not a very large variation - high was 4.01 billion and low was 3.64 billion. This part of Koch's statement would appear to be incorrect.
Tax Rates:
According to the Tax Foundation - link downloads .xls file, Koch is slightly overestimating overall Federal Income tax rate decline during the Coolidge Administration. In 1923, the top marginal rate was 58%. In 1929, it was 25% - that's a 57% decline. The bottom rate went from 4% in '23 to 1.5% in '29 - that's a 62.5% decline.
National Debt:
According to the Congressional Budget Office - again link is a .xls file, he is mostly correct regarding the US public debt. In 1923 it was 25.2% of the GDP ($85.3 billion) which works out to $21.5 billion. In 1929, those numbers were 14.9% of $103.6 billion, or $15.4 billion. That works out to about a 40% decline, if debt is measured as a percentage of GDP and a 28% decline is measured in dollars. Depending on how your want to make the calculations, he is either slightly overestimating the decline or slightly underestimating the decline.
Unemployment:
So the government didn't publish this information prior to 1947, so there isn't an "official" unemployment rate that I can find. But, according to an article by Christine Romer, entitled "Spurious Volatility in Historical Unemployment Data" from the Journal of Economic Policy, Vol. 94, No. 1 (Feb., 1986), pp. 1-37 - link is a pdf, the unemployment rate for 1923 was 4.8% and for 1929, it was 4.61%. If this data can be trusted - I would argue that, coming from an academic journal, it can be trusted - then Koch is simply incorrect on this point. Point of order though, an earlier method of calculating unemployment mentioned in the above cited article does give an unemployment rate of 11.72% for 1921. But there was a major panic after the end of the First World War, so that would explain the high number there. It's gone down by the time Coolidge takes office in '23.