I happen to be reading Adam Hochschild's To End All Wars right now, a book primarily about WWI but also containing some useful information about the Boer War, which was one of the last major testing grounds for the British military in the run-up to WWI. A character in a Rudyard Kipling story described it (reflecting his author's viewpoint, no doubt) as "a first-class dress parade for Armageddon."

As you say, the British had a much larger and more powerful army, and the war was not expected to take a long time. It was widely seen as just another breeze of a conflict quashing rebellious colonials, wot wot. One of the points Hochschild emphasizes is how old-fashioned and even condescending the initial British view of the Boer War was:

British officers talked of combat as so much sport. Men ordered to advance against Boer positions, called "beaters," were to flush the quarry from their hiding places as in pheasant hunting. A captain in the Imperial Yeomanry declared that chasing Boer horsemen across the veldt was "just like a good fox hunt." The first British commanding general in South Africa, the paunchy, double-chinned Sir Redvers Buller, ordered his soldiers not to be unsportsmanlike "jack-in-boxes" who ducked after standing up to fire their rifles.

There were a number of unexpected setbacks, but eventually the persistent Boers were crushed on the traditional battlefield: "the Union Jack now fluttered over South Africa's towns and cities, garrisoned by hundreds of thousands of tall-helmeted troops who outnumbered the remaining Boer fighters more than ten to one." However, these fighters had realized that even if they could not beat the mighty British on the battlefield, they could take a more unorthodox approach:

[The British army] found themselves pursing elusive, bearded warriors in civilian dress who refused to acknowledge that they had been beaten. Mounted Boer guerrillas raided British outposts and railway lines, ambushed British troops, and then disappeared into South Africa's endless plains. A proper cavalry charge, like that at Kimberley, was no use if you couldn't even find the enemy.

In so doing, the Boers had undercut the British advantage in manpower and firepower, instead relying on speed and mobility to disrupt the workings of the colonial system. The British were quite taken aback by this, and took ferocious action:

In response, the British decided to cut the roaming bands of Boer raiders off from their food and supplies. This meant that wherever the guerrillas attacked, British soldiers ruthlessly destroyed Boer farm buildings, crops in the field, and food stocks for dozens of miles in all directions. From some 30,000 farms, black pillars of smoke rose into the sky and flocks of vultures swooped down to feast on more than three million slaughtered sheep. [British] commanders ordered troops to cut down fruit trees and poison wells, to use their bayonets to slash open bags of grain, and to torch families' furniture and possessions along with their homes.

These families had to go somewhere, of course, and almost as an afterthought the British set up a series of concentration camps to hold refugees, including over 100,000 Boer women and children as well as Africans. (Hochschild hardly mentions the Africans displaced by the war. Wikipedia claims that "Although most black Africans were not considered by the British to be hostile, many tens of thousands were also forcibly removed from Boer areas and also placed in concentration camps.") Given the fact that the British hadn't really accounted for dealing with the needs of such a mass of people, conditions in the camps were horrible, as activist Emily Hobhouse documented:

At the first camp she visited, the heat was overwhelming, flies covered everything, and in the tents where destitute, traumatized families were living, the nearest thing to a chair was often a rolled-up blanket. In the chaos of being rounded up by British troops, she discovered, some of the Boer women had gotten separated from their children. The food was terrible, drinking water came from a polluted river, and up to a dozen people were crowded, sick and well together, into each tent.... Elsewhere, she saw corpses being carried to mass graves. "My heart wept within me when I saw the misery. (When a final tally was made after the war, it would show that 27,927 Boers—almost all of them women and children—had died in the camps, more than twice the number of Boer soldiers killed in combat.

TL;DR In some sense, then, the atrocities were just the outgrowth of British failure to foresee the tenacity of the Boers and the guerrilla warfare that would characterize the last stage of the conflict. Niall Ferguson writes: "This was not a deliberately genocidal policy; rather it was the result of disastrous lack of foresight and rank incompetence on the part of the military authorities."