I answered a similar question to this some time ago here, but I'll repeat it below with some additional information. Non-sequential numbering in seemingly sequential units sometimes arose naturally, out of specific regulations as I've described below, or it was used to purposefully confuse enemy intelligence; they could not describe the exact composition of all of one specific type of unit, what they were bound to face when they heard a specific unit or group of units was coming their way, or deduct exactly how many there really were, wither in the combat zone or preparing to enter it.

This is a...huge question. There's so many different variations from army to army and time period to time period. Every army has its own little quirks and practices. Sometimes numbers are chosen very deliberately and have a meaning, sometimes they're just products of how an army grew and evolved.

Sometimes, unit's designation might not be "rational," but the number contains important information about the unit's origins. For example, you might noticed that U.S. Army tank destroyer units have oddly-clustered numbers. The 601st, 767th, and 805th Tank Destroyer Battalions all fight in Tunisia in 1942, for example. There's some logic to this scheme.

In 1941, the U.S. Army had taken all the anti-tank battalions in the entire U.S. Army and re-designated them as "tank destroyer battalions." They were given numbers that reflected their origins - battalions from infantry division were in the 600s, battalions from armored divisions were in the 700s, and battalions from the the GHQ field artillery battalions were given 800-series numbers.

The Army tried to keep the 600-series units with their old infantry divisions (i.e. 601st Tank Destroyer Battalion with the 1st Infantry Division, etc.). Some the 601st to the 64th Tank Destroyer Battalions were matched with the 1st to 45th Infantry Divisions. However, this all got mixed up as units were disbanded or reassigned as needed. Winning the war was more important than keeping number straight.

Constantly re-numbering battalions to keep the neat numbering battalion-matched-to-division scheme would have created an organizational nightmare. Imagine this, the 601st TD Battalion is attached to the 1st Infantry Division when it orders some spare tires. The tires are despatched and en route. Before they can get there, the 601st is reassigned to a new division given a new number. Are the tires going to get to the right place?

The same scheme was also used in some air forces. For example, during WWII many RAF units were given numbers that matched their origins. No. 300 to No. 352 Squadron all had aircrew from countries in Occupied Europe, like the famous No. 303 Polish Fighter Squadron. No. 400 to No. 490 were all flown by RAAF, RNZAF, or RCAF crews from Australian, New Zealand, and Canada, respectively.

In other cases, the number contains important information about the units' role or branch of service. Going back to the RAF, No. 651 to No. 666 Squadrons were Air Observation Post (AOP) squadrons that flew as artillery spotters. And the Fleet Air Arm (originally under RAF control until 1939, when it transferred to the Royal Navy) was organized into squadrons with numbers in the 700s and 800s.

In other cases, unit designations were jumbled deliberately as part of deception operations. The Special Air Service got its start in 1941 as "L" Detachment, Special Air Service Brigade, even though the Special Air Service Brigade didn't exist and Detachments A through K were equally imaginary. Founder David Stirling had wanted to fool the Germans into thinking there was a much larger force of paratroopers. In reality, L Detachment only had 65 soldiers!

This deception could also be done on a much larger scale. In preparation for the invasion of Occupied Europe in 1944, the Americans and British launched Operation Fortitude, a massive deception campaign to fool the Germans of their real intentions. Of Fortitude's children was the phantom First United States Army Group (FUSAG).

One would imagine the First Army Group would be tasked with the important job of making the main landings in Europe (in reality, the job went to 21st Army Group). Allied planners certainly hoped the Germans would make that assumption. To further sell the deception, Allied planners created phony divisions like the British 2nd Airborne Division and the American 25th US Armored Division. They transmitted radio traffic and leaked messages to German intelligence to reveal the "existence" of these units

If you've ever wondered why are some missing U.S. Infantry Divisions with numbers in the 50s, this is why. The 50th, 55th, and 59th Infantry Division were all phony divisions "assigned" to various deception plans.

And yes, units do get shuffled. Imagine you have several matched sets of china in your kitchen. Blue in one cupboard, white in another, and so on. Since you're well-organized, you want to keep each set in its own cupboard. But then you have a lot of guests over - so you have to take down one set. A plate gets dropped and you have to replace it with one from another set. The party's over and your washing up, but you're in a hurry, so you put the plates away wherever there's room. By the end of the night, you have blue and white china mixed in one cabinet and red and white china in another.

That's how modern armies work and have worked. There's a crisis and the nearest units get sent out, regardless of number. A unit's been garrisoned in one place and it doesn't make sense to move them closer to a unit with a similar number. A unit gets wiped out and has to be replaced with another one. And so on.

​