Egypt might have been the earliest "civilisation" (city-dwelling culture) to use celestial navigation at sea. At the same time that Egyptians were voyaging across the eastern Mediterranean, the Austronesian maritime expansion into Micronesia and Melanesia, and later, Polynesia, was under way. Long-distance Egyptian navigation at sea predates the earliest known similar feats by Minoans and Phoenicians.

However, it is extremely unlikely that either were the first to navigate by the sun and stars. Celestial navigation is used by many traditional cultures for overland navigation - within one's own territory, familiar landmarks can be used, but when one goes well beyond familiar territory, the sun and stars are often the main resource. For example, multiple groups in Australia use the stars for navigation, and so do the San (Bushmen) of the Kalahari. Celestial navigation is so widely-spread that either (a) it was discovered before the settlement of Australia and the Americas (and possibly before Homo sapiens left Africa), or (b) it's been repeatedly rediscovered, which also suggests that it's been known for a very long time (and long before Egyptian civilisation began).

Also, seagoing long predates Egyptian civilisation. Whether or not early seafarers used the sun and stars for navigation is unknown, but since celestial navigation was used on land, it's likely that they also used it at sea.

Whether any of that counts as "modern navigation" is a matter of definition. Even before the inertial compass and GPS, modern navigation used accurate clocks, magnetic compasses, and celestial navigation techniques developed in the 18th and 19th centuries. What the Egyptians did was not this. However, if we generously call any instrument-assisted celestial navigation "modern navigation", then the Egyptians might be the first. As an early seafaring civilisation, and a civilisation with early surveying and the use of astronomy for accurate direction finding, they had all of the ingredients to use instrument-assisted celestial navigation at sea, and probably did so.