To answer your question directly; yes, the concept of a secret police did exist in the ancient world, though in more limited scope than what you'd think of when imagining more recent Secret Polices such as the Soviet NKVD and the OGPU they worked under. I'm nowhere near knowledgeable enough to talk about many that existed, but I'll focus on the Roman Empire's Frumentarii.

The Frumentarii during the early Roman Empire were originally collectors of grain to be shipped from the farms to the city markets, essentially a traveling tax collector for farmers and other estates. During the 2nd Century AD Emperor Hadrian sought to create a information network for the far corners of the Empire, but the Speculatores (Scouts) of the Praetorian Guard were simply not up to this task given the limited scope of the Emperor's guard. The Emperor turned to the Frumentarii, who possessed the most thorough knowledge of land and settlements throughout the empire through their grain collection duties, and employed them to act as an informant service to nearby urban police.

As far as I know, the Frumentarii were not authorized to make arrests themselves, but collected information from the surrounding areas and passed it on to local police in order to make arrests.

The Frumentarii were generally despised by the Roman Populace however, and were eventually disbanded by Emperor Diocletian in the late 3rd Century due to public outrage towards their activities.

For further reading, I highly suggest Intelligence Activities in Ancient Rome: Trust in the Gods but Verify by Rose Mary Sheldon, where she goes more in-depth on the Frumentarii and their activities.

This is simply the one secret service that I am knowledgeable enough to talk about in-depth, I'm sure others will have sources on other intelligence services in Egypt, Carthage, or the Successor States of Alexander's Empire.