It's quoted in his mistress' diary which is considered accurate by historians. The relevant entry (9 March 1934) reads:

Debate last night in the House on Air—strong demonstrations in favour of increased no. of fighting planes. [Lloyd George] says it could have been avoided but for Simon's mismanagement. At Geneva other countries would have agreed not to use aeroplanes for bombing purposes, but we insisted on reserving the right, as [Lloyd George] put it, to bomb niggers! Whereupon the whole thing fell through, & we add 5 millions to our air armaments expenditure. But [Lloyd George] is not yet ready for his attack on them. He would rather wait until the autumn.

Lloyd George was a Liberal and at this time in his career was in favour of disarmament. He believed that it was wrong for Germany to be disarmed while Britain and France were armed and he agitated in favour of disarmament. He believed it was the British government's fault that this disarmament proposal failed. He is clearly describing the government's attitude in unflattering terms.

Source: A. J. P. Taylor (ed.), Lloyd George: A Diary by Frances Stevenson (London: Hutchinson, 1971).