There are a few comments one can make here.

One is selection bias. Byrson works in the UK and writes for an Anglophone audience and I suspect a French or German author — much less a Chinese one — would accord different levels of detail to other countries' scholars. The French and Germans have at least as large a claim to the development of European science as the British do (and there are plenty of other key countries in this, too, depending on the time period), and focusing the development of science exclusively on Western Europe is already a somewhat "warping" view of things. The British certainly had a big influence from the early modern period through the modern period, but if you're getting the impression that nobody else was making similar strides, that's just because he isn't talking about them.

Second, that aside, the "top tier" scientific powers of the early modern period were the British, the French, and the Dutch (with shout-outs to the Italians and those in the Germanic states). Why was that? Partially because of state sanctioning and funding of learned societies, partially because in these states at this point in time there were complex overlaps between the worlds of knowledge-production, politics, religion, and commerce. What we today call "science" was one element in a rapidly changing world, and it is not a coincidence that these were all imperial, colonial powers, either. Science did not show up in a vacuum — it was a context of shifting power, and scientific knowledge increasingly became seen as part of the engine that shifted it (although it did not really pay great returns for about a century).

For the more modern period (e.g. late 19th/early 20th century), Britain, France, and Germany were the "top tier" by far, with Russia, Japan, and the USA in a "second tier" behind them. In each country there were different dynamics of how science worked. In all three countries, science had "professionalized" to a strong degree, meaning that there were now well-defined paths to starting a career in it, finding support for the work, and so on. The UK in particular adopted educational reforms and requirements that emphasized particularly mathematical approaches to science (the Tripos examination system) which is one of the hallmarks of British contributions (as opposed to, say, the Germans, whose linkages with industry and emphasis on applied work characterize their science). All of these nations had increasingly literate populations, lots of capital, robust university systems, and huge economic and political ambitions.

All of which is to say, one can have a few stand-out scientists without much infrastructure and support. There are bright, exceptional people of all nations. To be a scientific "powerhouse," you have to have the means of finding many interested people (most of whom will not be "geniuses" by any measure), training them (the role of autodidacts essentially ended by the 19th century, as the work became far more advanced and specialized), and getting them the resources they need to continue their work (whether it be in a university system, a system of patronage, linkages with industry, whatever). And you also need to keep them, which is one of the reasons that Western Europe in general became totally surpassed by the United States during and after World War II, when the latter became (finally) convinced that science was worth supporting in a big way, and not only ended up accepting a large emigre population, but created an infrastructure that could attract and retain world-class scientists from anywhere.

The above is generalized from many studies in the history of science. For a close look at the dynamics of the physics community in particular, I would recommend Helge Kragh's Quantum Generations — it is an excellent book for this kind of question, because Kragh is explicitly comparative when it comes to different national programs, and never falls into the false pit of thinking "genius" plays a large role in any of this.