It might be worth reading the book to check what it actually says - your professor might be (probably is) overstating things. For much of history there was certainly no such thing as a 'no-pursuit' rule, and it is very unlikely that military powers would have formally codified such a rule - the pursuit, after all, is where the most significant casualties can be inflicted on the enemy.

While it might have held in the 18th-century period of 'civilised' or 'scientific' warfare, it most certainly did not in many other periods - for example, in the Napoleonic Wars, the pursuit after a battle was often more crucial than the battle itself. After Jena-Auerstadt in 1806, the French Army near-completely destroyed the Prussian Army, advancing from the river Saale (where the battle was fought, in modern-day Thuringia) to the Oder (the modern-day German-Polish border) in 23 days and capturing 140,000 Prussian soldiers and 200 cannon.

It's also worth noting that victorious forces might choose not to pursue a defeated foe for reasons of their own - like being exhausted, overextended, or shattered by casualties taken in their victory.

There might be a scintilla of truth regarding ritualized warfare as a means of settling external disputes in some societies, but I don't think the evidence supports any idea of this applying to Early Modern Europe. The idea would imply that a battle would resolve the dispute and thus end the conflict, thus shortening the war. A cursory glance at the military history of Early Modern Europe would dispel this notion immediately.

The dominant form of warfare for three centuries, from the sixteenth to the nineteenth century, was not the pitched battle. Rather, it was a constant campaign of attritional sieges and often brutal raids and reprisals. Armies sought to wrest control of key geographic positions from their rivals, whether by maneuver or systematic battery and assault. Much of an Early Modern army was not formed up in the line of battle, but rather dispersed to garrison the dozens of fortifications necessary for projecting power into the seat of war and protecting the home state's interior. During the Dutch Wars of Independence, the Spanish Monarchy put their best troops in their garrisons, since it was understood that these small detachments, engaged in constant raids, skirmishes, and executions (destroying towns that failed to provide the contributions demanded by occupiers), required more skilled and motivated soldiers than the relentlessly drilled automatons formed up in the rank and file.

Even when pitched battles did occur, the results rarely lent any credence to the idea that they could resolve disputes. There were very few last battles in Early Modern European wars; Frederick the Great fought a great many battles in his military career, winning some, losing others, but neither his most brilliant triumph at Rossbach nor his most devastating defeat at Kunersdorf brought the war to a close.

What's interesting to me is that the idea of a 'no pursuit rule' and a 'verdict of battle' don't seem to sit in the same scale; surely, to have a practical verdict of battle, the enemy army would have to be unable to contest it, and the only way that could happen is with a vigorous pursuit. Fortunately for the defeated army, this rarely happened, but it wasn't because of any kind of agreed upon rule.

The most important factors limiting an army's ability to pursue a defeated opponent were operational, not customary. In the 18th century, it was difficult to force an opponent to stand and fight a battle in the field if they didn't want to. Most states possessed a fortified frontier that could keep an invading army out of its core territories; grinding through the outer ring of fortresses would often chew up the bulk of an army's offensive strength by itself. Furthermore, since armies in the period generally maneuvered as a single piece, and often took hours to move from marching column into battle formation, it was difficult to catch an army that didn't want to fight, since they could retreat in any given direction. Thus, an army would only accept battle when they felt they had an advantage that justified the risk, such as occupying key ground or having superior numbers. Each soldier was very valuable to his employer, as it took years of training (and feeding, and clothing, and equipping) to produce a good musketeer, and battle chewed through them at a prodigious rate. When Frederick the Great sought battle, it was because his kingdom lacked a good fortified frontier, and thus it only had walls in the Spartan sense, the ranks of its soldiers

An army deployed for battle typically fought in two lines, each three ranks deep, with artillery interspersed, and cavalry fighting on the wings. This made battles quite symmetrical; with infantry blasting each other on the firing line, while a more disciplined force might eventually persevere to triumph, it was likely to lose a lot of men in the process. A good example of this is the aforementioned bloodbath at Kunersdorf. Both armies lost between a third and half their men in a single day; at first, Frederick considered it the end of his kingdom, but the Russian-Austrian enemy was so crippled that they had to pull back to lick their wounds, rather than march on Frederick's capital and destroy Prussia for good.

Furthermore, a defeated army generally had a fortress at its back, which both served to funnel supplies to an army retreating along a shortened line of supply, and as a physical refuge to preserve it until a counterattack could be mounted. Meanwhile, the victorious army attempting a pursuit would find its line of supply lengthening, thus slowing the march and allowing the enemy army to more easily break contact. A good example of this is Frederick the Great's invasion of Bohemia in 1757, where after crossing the mountainous frontier, he combined the four separate columns he'd used to get through the passes into one army to attack the Austrian army in front of Prague. It was bloody, but he drove them into the fortifications. Meanwhile, though, the Austrians amassed an army of relief, which landed a hammer blow on Frederick's army at Kolin, which had only half the strength available to fight as it had just over a month before.

That's not to say there weren't agreed upon laws and customs of war, far from it, but they weren't laws that compromised military effectiveness. While we think of laws of war in terms of internationally signed treaties, what really governs war is the enemy's capacity for retaliation, and a mutual understanding of military necessity. Really, you can do whatever you think is necessary to win, -no one wants rules of war that make victory impossible- but you have the understanding that the enemy will do that to you too, so while some measures may seem to promise reward in a vacuum, on net, they may not be the best idea.

During the American Civil War, the Confederates threatened to hang every officer they found leading black troops for 'inciting a slave revolt'. Since the U.S. had thousands of Confederate prisoners, they threatened to retaliate by hanging an officer of equal rank. When the Confederates stated using black PoWs to build field works under fire, the Union General Butler employed a corresponding number of Confederate prisoners within range of their guns. The common practice in 18th and 19th century wars was to exchange prisoners, and if their was a difference in the number available for exchange, to give them a parole, in which they promised not to fight until exchanged on pain of death if recaptured. This was mutually beneficial, since neither side wanted to supply men who weren't going to be fighting on their side, and as long as both sides observed the laws and customs of war, the system worked.

Even before the age of mass politics and republics, you see states putting everything they have over a period of years or even decades towards a final victory over their opponents, in which no military effective approach was off the table.