I think I've found the relevant passage, and she does sort of expand on it. She talks a couple times about how the British Empire was acquired in 'a fit of absent-mindedness'.

'It has often been said that the British acquired their empire in a fit of absent-mindedness, as consequence of automatic trends, yielding to what seemed possible and what was tempting, rather than as a result of deliber­ate policy,' is how Arendt puts it in the preface to part two. So, instead of the British Government decreeing that Britain will build an empire, and setting out to grab all the land it could, the British Empire developed more as the result of the British grabbing a piece of land here, because they could, a piece of land there, to defend a trade route or some other strategic concern, and then, one day, they woke to find that all of India and much of Africa was painted pink on the map.

This isn't an entirely false view of how things developed. There was never really a British government that had a stated policy of expanding empire for empire's sake, certainly not during the Second Empire [1], which I am more familiar with. Expansion did often happen on the wings of commercial interests, whereby British traders arrived, and then asked for help from the British government to defend their interests a little down the line. Most of the conquest of India technically happened under the auspices of private enterprise; the East India Company. The British Raj, which created an office in Whitehall for the administration of India, did not come about until the company was nationalised after the Indian Mutiny nearly kicked the British out of the subcontinent [2]. A lot of the Scramble for Africa on the part of the British is, arguably, a similar process, in that competition from the other colonial empires forced the British to choose between formalising the control they had of much of Africa via traders, or losing it to Germany/France/etc.

This does lead me to my point; Yes, the British government, rather than actively setting out to conquer these lands, did more often find itself having to choose between taking over, or potentially letting the local population harm the interests of British business. However, they still had the choice. The UK government consistently chose, in conflicts between Britons and local populations, to subjugate the local population and expand the empire. They did not get blackout drunk and wake up the next morning holding dominion over palm and pine. They actively made the choice of empire over harm to British business or prestige, and British imperialists, even if they were not officially backed by government, had faith that, if push came to shove in the areas they were making private land grabs, London would come to their aid.

That debate has largely been decided. You'll be hard-pressed to find a historian that says British government and imperialists built the empire 'absent-mindedly'. The debate today is more about the extent to which the average Briton cared. Did your average punter have a great attachment to the British Empire, or could they not care less whether John Bull painted another part of the map red? That could be a whole other answer, so I'll stop here [3].

TL;DR: I suspect this is about how the British Empire wasn't planned centrally in Whitehall, but more often the result of choosing empire whenever events in the world offered a choice of expanding it, or else have British business or prestige lose out.

[1] - That is, the empire after the independence of the United States.

[2] - The nuances of an 18th Century colonial monopoly company vs a modern multinational corporation make this somewhat different from Amazon hiring private armies and the US government taking over after the local population nearly succeeds in driving the company out.

[3] - I, personally, would probably say that the average Briton didn't actively think about it, but, as in government, whenever it came up, public opinion seems to have been on the side of empire (cf. Gordon of Khartoum).

Whilst u/TheBobJamesBob has done a bloody fantastic job of unpacking the question and explaining how the British Empire was not the result of some "imperial design" from Whitehall, I want to delve a bit deeper into some of the specific cases in each of the various regions which the British government (either willingly or reluctantly) chose to support their "men on the ground" and how that situation came about.

"The British Empire has hitherto been not an empire, but the project of an empire; not a gold mine, but the project of a gold mine."

Adam Smith writing in 1776

This sentiment towards the amalgamation of territories, "zones of influence", "spheres of domination", and a litany of other catch-all terms for the lands which (in one way or another) were impacted by Britain may have softened a century after Smith's writing, but it still rang fairly true. London had not set out in the late 17th century to conquer and coerce for itself the various polities, nation-states, or indigenous populaces which it eventually did, but it was often the arbiter (or to indulge some Latin, the ultima ratio) of what would become imperial rule. The government in London usually had little to no control over what the "men on the spot" did with the initial British bridgeheads, and this was both due to the long communication times (weeks if not months in the age of sail), so they left the initiative up to these men. Even the motives of the so-called "empire builders" differed vastly, whether that be Robert Clive in India (known as Clive of India), Cecil Rhodes in Africa, or the various companies which represented the British interests in the region. More often than not, economic reasons lay at the heart of expansion; though we must stress, not necessarily territorial expansion, but rather the expansion of influence and business. So where did the British government back in Westminster factor into all of this, seemingly independent and detached, empire work?

The first role which the British government played was simple: protection. In the early decades of British empire-building there was almost always a rival government (European or otherwise) which also had its agents in the area. British merchants in India always preferred the officialdom which surrounded a "royal emissary", helpful when dealing with Princes or other nobles who were much more powerful than mere "men of trade" on the continent. During the scramble for Africa, it was not uncommon for governors to enlist the help of the Royal Navy or (with even more difficulty) the British Army. It was under the guns of HMS Prometheus that the Island of Lagos was "peacefully and willingly" ceded to the British in 1861, with its King Docemo later complaining that had he not signed the cession treaty, the commander aboard Prometheus would have "obliterated the Island in the twinkle of an eye". It was always helpful to use the threat of force, rather than force itself, to persuade a rival party to come to the negotiating table.

The second role which London played in these ventures of imperial expansion was economic security. By granting practical monopoly rights to certain land or resources, the government at home was assisting in warding off rival claimants to economic goods and capital (note that those rivals could be foreign, local, or even fellow Britons in nature). It was also a benefit to the government however, which in the pre-industrial mercantile age knew that taxing trade was an excellent route to profit. As early as the 1650s, the British government attempted to harness this source of revenue strictly for themselves in what were the Navigation Acts. Any valuable products from overseas holdings were to be shipped to British ports first, where the merchants would pay duty and then re-export it to the intended market. The Acts also demanded that any such trade be carried on British ships manned by British crew. This not only ensured that the main profits of trade flowed to Britain, but also that colonial producers would have to rely on the British suppliers back home for any materials they had to import. John Darwin on this economic link:

"By the mid eighteenth century, government, trade, and the exertion of sovereign power were laced together in a huge vested interest. For its part, the London government expected that an East India Company grown fat on its monopoly trade would supply it with loans. The fear of its failing, and bringing down the rest of the City, became a critical factor in ministers' Indian policy by the 1770s."

As an aside, TheBobJamesBob alluded to how the British in India, up until the Sepoy Mutiny of 1857, was largely represented by the British East India Company. If you are further interested in this matter, u/conqueror_of_destiny and myself go into more depth on it here.

Even in the late 19th and early 20th century, the Colonial Office back in London was a motley collection of ministries, administrations, and sub-divisions all responsible for various parts of the empire. Taking the initiative from Whitehall directly was extremely difficult, though not quite impossible, under these conditions. One had to get the various authorities cooperating with one another (not so easy a task), the Treasury loathed overseas spending (dismissing some requests for colonial capital with a single word), and the War Office alongside the Admiralty jealously guarded the "reserve currency" of British power (hence the critical need for British colonial administrators to recruit local troops and auxiliaries into their battalions).

In sum then, I shall echo the argument of the previous answer by remarking that the British Empire may not have been the result of an "accident" in the "by Jove, when did we take over that bit of the world?" sense, but it was certainly not the result of some "imperial master-plan" constantly being issued from Whitehall to the various men on the spot across the bridgeheads in various continents. There was always a link to London to be sure, but until the advent of technologies which shortened communication times and the necessary infrastructure to impose (with varying success and discipline) "direct rule" from parliament, the men on the spot were the agents whom London had to trust (and at times, bail out) to expand British interests in the region. John Darwin, an imperial historian whose works on the British Empire I highly recommend for further reading, writes:

"They [the private imperialists] regarded themselves as much the best judge of the local political sense: their men on the spot would know how to deal with any difficulty there. But in their charters and patents they had to acknowledge the ultimate authority of the government at home over their local activity. They might have been private imperialists, but they also became in law the agents or empire: over what they possessed on the ground was extended like an umbrella the sovereignty of Great Britain."

Sources

Darwin, John. The Empire Project: The Rise and Fall of the British World-System, 1830-1970. Cambridge, UK: Cambridge University Press, 2014.

Darwin, John. Unfinished Empire: The Global Expansion of Britain. New York, NY: Bloomsbury Press, 2013.

Jackson, Ashley. The British Empire: A Very Short Introduction. Oxford: Oxford University Press, 2013.

Smith, Goldwin. "British Empire in India." The North American Review 183, no. 598 (1906): 338-48. Accessed February 11, 2021. http://www.jstor.org/stable/25105620.

Wilfley, Lebbeus R. "How Great Britain Governs Her Colonies." The Yale Law Journal 9, no. 5 (1900): 207-14. Accessed January 3, 2021. Accessible online (free) here.