Until the 18th century, borders were often only demarcated in a few areas (and then only in Europe and a few other places). So the Tweed marked the north-east border between Scotland and England, the Rubicon the boundary between Italia and Cisalpine Gaul on the Adriatic side and so on, sometimes with a customs post on major roads. Lacking a convenient sea or river, borders kind of faded out - this town might be recognised as belong to France, and this other to the Empire, with in-between left to local influence. Political arrangements mirrored this - the core area would be governed tightly, then a ring of less-tight governance, then again an area where the state exacted broad obedience (maybe aid in war and freedom to trade), then another where obligations were vaguer. China maintained garrisons along the Silk Road for long periods without much control beyond the road itself, Rome had a network of client rulers beyond the Rhine and Danube, British control of the north-west frontier extended to the major roads, a few forts and periodic reminders of its military predominance.

So the maps are misleading in their precision. Historians use written accounts from all sides, archaeology (eg the Roman limes), boundary markers and other indicators, but there is often a margin of error.

Generally, rivers and mountain ranges demarcated borders. It depended on the region. For example, in the Middle East, borders were somewhat irrelevant. When you read about the Roman vs. Persian Wars between ~55 BC to ~700 AD, Roman and Persian units would "cross" over the border.

However, the borders were usually the Euphrates River. North of the Euphrates were a series of mountain ranges that demarcated the Armenian Kingdom versus the Roman and Persian Empires. Over the centuries, as the proxy kingdoms were annexed by the two rival empires, these mountain ranges became more important. The main mountain range would be the Zagros Mountains (currently the border of Turkey and Iraq, as well as the border of Iran and Iraq and the border of Azeribjan (Nakhchivan Province). Often, the borders would move and other mountain ranges would constitute the borders. These mountain ranges would variously be the Armenian Highlands or further west near modern-day Ankara (ancient Angora).

Noting the mountains below the Zagros Mountains, sometimes Roman sources conferred modern-day Sinjar (ancient Singara) as one of the borders between the Romans and Persians.

So, as you can see, these "borders" were more like markers than anything else. Taxation by central governments in these areas was not common. It was hard to maintain a series of outposts to essentially perform the same duties as modern-day border gates (like Ibrahim Xalil which is the only border gate between Turkey and Iraq). Modern-day border gates are a major source of revenue for countries; either taxing imports or taxing exports, as well as inspecting goods, and so on.

While tax and customs inspections did occur during the Roman Empire, much of this happened in major ports like Alexandria. The extremely long borders were not amenable to such extreme oversight by central governments.

Additionally, central governments would use nomadic tribes to "patrol" the border. However, over the course of history, these tribes would often revolt and therefore the idea of a "border" was somewhat mute. For example, Shah Qulu revolted. Many of the border tribes joined his revolt. These tribes, furtherest from central government, did not have the same ethnicity or religion as the interior people. Despite being tasked with "guarding" the border, they instead controlled the border by threatening to join another empire or simply backing a candidate and causing a coup. It was not until early-modern history that most of these tribes (various Kurdish, Armenian, Assyrian, and Arab) across the Middle East would be forced into towns and the borders heavily patrolled by professional armies organised by central governments across the region. This, of course, has been aided by advancements in technology like drones, cameras, and so on. However, even China still uses Tajik men to patrol their border with the Tahiks and Afghans.

So, as you can see, the idea of "borders" was more a way to denote an idea of where the limits of your power extended. Borders, which for thousands of years, were patrolled by nomadic tribesmen, have been heavily consolidated and now are patrolled by professional armies/police forces. In the ancient era, the idea of a border was more like, "don't go there, we lack influence there," and therefore the idea of the interior versus the exterior of an empire. But, there are caveats: empires which maintained strong navies and had well-established ports were able to still tax ships moving from port to port. And maritime borders were often far closer to the modern-definition of a border than land-borders in the ancient-era.