Generally speaking, the limites were a bit different from modern borders. First off, they existed only for limited time (ha!) and in specific parts of the empire. It is more useful to think of them as a network of border/customs checkpoints combined with a network of early warning sites for possible barbarian incursions that could alert troops from the interior of the province to react to threats, and also discourage smaller raids than as a modern border system. There were communities of translimitanei, people living beyond the limes that were still Roman citizens and subject to Roman jurisdiction.

With that out of the way, it is easier to look at ancient Roman aministrative areas as which subunits belonged to it, that is, which smaller units belonged to a province, which towns, manors and villages belonged to these smaller subdivisions and so on than to look for clean borders, because you will often be hard pressed to find exact borders. Sometimes, they can easily be reconstructed by looking at modern ecclesiastical dioceses, which often conform nicely to ancient territorial subdivisions. At other times, it is not as easy, and often times it can only be said that the border must have been between point X and point Y, which both contain references to the respective different provinces or civitates or what have you. Sometimes, there are obvious geographical features that are then said to be the provinces border, sometimes, not.

Now, how would a traveller have known in ancient times? There are several possibilities. For one, there was a system of tolls between clusters of provinces, such as the Gallic provinces, these tolls would have been levied near the borders between the provinces. Oftentimes, there were also interior control stations manned by special military veterans that would supervise traffic.

Another possibility was to look at milestones. Those milestones usually counted the distance to the so called caput viae, the head of the road, where the road began. This would usually be the capital of the province or of the smaller subdivision, it could also count the miles to or from Rome, or to the province's border.

So a traveller usually would know either because a) a post challenged him to ask him if he had to declare something b) the name of the city to/from which the milestones counted changed or c) the milestone told him/her exactly how far it was.

Boundary stones in the modern sense usually only marked the limits of plots of individual plots of land, so they re only useful to, for example, reconstruct the territory of a city and its land.

How did the geographers and bureaucrats know the scope of geographical area that belongs under his jurisdiction?

They knew which administrative centres of the lower level belonged to their jurisdiction, who in turn knew which towns, villages and manors belonged to their jurisdiction and so on. Roman borders were usually established between groups of people, so what was important was the links between certain 'nodes', more so than the exact borders between these areas.