I can't speak for every example, but the certain examples like the Japanese transition from 6.5mm to 7.7mm and the NATO light rifle trials offer some insight into the thinking behind the selection of .30 caliber. Most of the arguments boil down to effective ranges - .30 caliber seemed to be the sweet spot for relatively flat trajectories but longer range.

The Japanese were one of the few powers to adopt a 6.5mm caliber, but their experience in China led to several complaints about the power of the cartridge, especially at long distances. While the cartridge is very flat shooting and perfectly acceptable in retrospect for the distances that combat ended up happening at, its limited range compared to heavier cartridges being encountered in China (such as 8mm Mauser) was a major limitation on heavy machineguns. Following the doctrine established in WW1, heavy machineguns were an important source of relatively short range semi-indirect fire. Firing at maximum ranges, heavy machineguns would be used to create a beaten zone up to 2-3 thousand yards away on an arcing trajectory. Extending the effective range of the heavy machineguns was a fairly important prospect - so much so, for example, that the US ordnance department would perform a fair number of tests to revise the bullet of a standard 30-06 cartridge from a flat-base to a boat-tail design in order to increase the maximum range. Looking at Japan during the interwar period, they would first employ 7.7 mm cartidges in heavy machinegun roles. The Imperial Japanese Army would scale up their Type 3 heavy machinegun to the Type 92 in 1932, and both the Army and Navy would adopt a 7.7mm cartridge for their aircraft machineguns around the same time (the Navy opting for a copy of .303 British). The refinement of the Army cartridge into what became 7.7x58mm took several years, and it wasn't until 1939 that Japan had a more complete complement of infantry weapons (Type 99 rifle and machinegun) to fire the new cartridge.

As far as I understand, Italy's motivations for their late adoption of a 7mm cartridge on the eve of WW2 were similar, though the limitations of their 6.5 caliber were made worse by the design of the Carcano's rifling making it impractical to shift from obsolete bottlenose cartridges to the pointed spitzer bullets in use by every other power.

​

For the NATO light rifle trials, the cartridge selection was driven largely by US Army Ordnance strong-arming the rest of NATO. The driving motivation again was range, but this time at more reasonable distances. Heavy machineguns had largely been replaced in their indirect fire roles by mortars around this time, and the focus instead was on effectiveness at relatively long direct-fire distances. A key requirement from the US was the ability of the cartridge to penetrate helmets at something like 500 yards, which all but required something with the same power to it as 30-06. Similar trajectory requirements forced high velocities and heavier bullet weights, all of which worked against the more lightly powered .280 British cartridge. While .280 British was a bit of a middle ground between current intermediate cartridges and full-powered rounds like 7.62 NATO, it performed well in the closer distances the British had designed the round for. But at the longer distances the US wanted the round to be effective at, the .280 round lacked the energy to penetrate helmets and had a more arcing trajectory that created a "safe zone" that was deemed unacceptable.

For the Soviet 7.62x39mm intermediate round, it seems likely that they got the idea to scale it down from the existing 7.62x54r from the 7.92mm Kurz in use in the early German assault rifles like the MKb. 42(H). They'd been experimenting with 'intermediate' rounds since WW1, but the wartime expediency of scaling down an existing round made a short 7.62 attractive. Monetchikov's authoritative history of the development of Russian assault rifles has this to say in Chapter 3:

During the development of the new “intermediate” cartridge, 314 variants of different samples were calculated, of which 8 were recommended for further research. After a series of experiments, due to insufficient efficiency and a decrease in the lethal effect of bullets, all projects of a caliber of less than 7.62 mm were rejected... The creation of a 7.62x41 “intermediate” cartridge in 1943 served as the basis for the beginning of promising work on the development of qualitatively new models of domestic individual automatic infantry weapons. Shortened compared to a rifle. an automatic rifle cartridge with a flangeless sleeve made it possible not only to significantly simplify the methods of delivering cartridges in automatic weapons, but also to increase the firing rate with a decrease in recoil, although big claims were expressed (including V.G. Fedorov) regarding the new bullet, and. in particular, to its much lower gyroscopic stability in flight with respect to the light pool of the 7.62 mm rifle cartridge of the 1908 model.

The “intermediate” cartridge of the 1943 sample consisted of a bimetallic sleeve 41 mm long, without a flange, with a smaller diameter (than the sleeve of the rifle cartridge of the 1908 sample) with a capsule; a charge of pyroxylin gunpowder of a special brand VU (rifle, shortened cutting); a bullet modeled on a 1908 light bullet with a lead core and a cylindrical tail.

The new cartridge was intended for the destruction of open (or covered with small obstacles from local objects) living targets at ranges up to 1000 m. Providing targeted fire up to 800 m with a direct firing range of 325 m. A bullet with a lead core to a cartridge of 1943 model weighing 8.0 g (for comparison, a rifle model of 1908 - 9.6 g, a pistol model of 1930 - 5.5 g) had an initial speed of 740 m / s (respectively, a rifle model - 865 m / s, a pistol model - 520 m / s) with a muzzle energy - 229 kg / m (rifle - 362 kg / m, pistol - 75.7 kg / m). Compared to the rifle, the new cartridge of the 1943 model made it possible to increase their stock by 1.38 times (the mass of 100 rifle cartridges is equal to the weight of 138 cartridges of the 1943 model) and to lighten the weight of the weapon. An important factor was the production factor.

(Machine translated because it is late, I am lazy, and I'm still nowhere near fluent in Russian.)