As far back as the Book of Documents (尚書) or the Daodejing (道德經), we can find references to the “five colours” (五色), which is what Sunzi is referring to in this passage. These colours (green/blue 青, red 赤, yellow 黃, white 白, and black 黑) have been traditionally understood to be the primary colours because of their affiliation with the Five Phases (五行): green/blue was associated with wood, red with fire, black with water, white with metal, and yellow with earth. When the Art of War was written, the Five Phases had not yet fully matured as a conceptual system, but pre-Qin and early Han dynasty texts do contain a variety of references to these groups of five colours (五色), five tastes (五味), and five notes (五音), that Sunzi is talking about in the passage you’ve cited.

The reason why these five colours are singled out, then, is due to their primacy in the “Five Phases” conceptual scheme, as opposed to the placement of these colours in a spectrum or colour wheel.

From a different perspective, these specific color terms / distinctions have been thought to emerge first in terms of stages of color names in a language (Berlin and Kay, 1969 / 1991), albeit with blue eventually being replaced with "grue" (green and blue) in later iterations of this theory (Kay, 1975). So for example, the original theory was that languages with few color terms would have the categories black and white first (and maybe gray), red next, followed by yellow and green and blue / "grue". It should be noted many of these ideas re universality of color naming categories across cultures / languages, are rather heavily disputed (see, e.g. Levinson 2000 for a review). I'm a vision scientist by training, so perhaps someone with a linguistics background who is more familiar with the contemporary status of this theory can chime in.

Not an answer to your direct question, but you may be interested in knowing he was actually correct. Primary colors mean the colors which can be mixed to produce all other colors. RGB is the primary colors in a system where you mix colored light, like in sunlight or in a computer monitor. By mixing light in these three wavelengths, you can get any color - including white (all wavelengths) and black (no light at all).

But Sun Tzu is talking about the coloring system where you create colors by mixing pigments, like in painting or printing. Pigments work by absorbing specific colors (wavelengths) from the light and reflecting the rest. This leads to a system with different primary colors, called the substractive color system. In modern terms the substractive primary colors are called cyan, yellow and magenta. Cyan is bluish and magenta is purpelish, but they are not the same hues as the blue and red in the RGB system. But due to the physical properties of pigments you cannot mix colors to get white and black, so you also need these two as primaries. If you look into a typical color printer you will se four inks: cyan, yellow, magenta and black. White is not needed as a separate ink because we print on white paper, but it is still a primary.

Sun Tzu would not be able to hit the exact primary colors though. The color magenta is not exactly red but a purpleish color which is not found as natural pigments, and it is only with modern chemistry (from around the 1860s) it have been possible to synthesize it consistently. So even if pre-modern people might have understood the theory behind primary colors, they would not in practice have been able to create all colors from just five pigments.