The Romans and the Persians fought many wars, but there were also long periods of peace. Your suggestions for why they should not have fought against each other can also be applied to every other polity in human history - yet conflicts were omnipresent and co-operation was often spurned in favour of self interest. Power, prestige and material gains were powerful motives indeed! Wars were also fought for a reason by rulers who thought they had a realistic chance of victory and were definitely not pointless. I'll talk mostly about the wars in the sixth/seventh century, but the points I make should hopefully be applicable for earlier conflicts too. Essentially, wars were fought to gain specific advantages and from my perspective each war ended with at least one side being satisfied. Realistically, there was only one war between Rome and Persia during which they even contemplated the destruction of the other empire, and it was something that only became a possibility years into the war, rather than something they planned for from the beginning. Nor should we see war as the 'norm' between the two powers, since there were many less dramatic instances of cultural exchange that occurred throughout late antiquity - M. Canepa's The Two Eyes of the Earth, Art and Ritual of Kingship between Rome and Sasanian Iran (2010) is by far the book on this and I thoroughly recommend it, even though it is a bit academic in its style.

Aside from two brief periods of contention, the two empires were actually at peace throughout the fifth century, mostly because they had other threats to deal with - the Romans had to deal with internal conflict and the dissolution of the Western Roman Empire, whilst the Sassanids dealt with the Hephthalite Huns, which intervened regularly in Persian politics, killing one shah and imposing another, Kavadh, on the throne twice. They played a critical role in the next period of conflict, as it was provoked in 502 by Kavadh to get the funds necessary to pay off his Hephthalite sponsors. The Romans seemingly had a long-standing agreement with Persia to partially fund the defence of the Caspian Gates in Persia against steppe nomads that they abrogated in the late fifth century, so there was a genuine bone of contention between the two great powers at this point. Kavadh therefore had both a motivation and a pretext for war. The war was quick and ended up with a stalemate (with the Romans perhaps slightly worse off), but Kavadh did get what he wanted, payment from the Romans, so it was a perfectly valid policy on Kavadh's behalf.

Kavadh however re-opened the war in 530, but again he had valid reasons. So did the Romans actually, as they nursed grievances from the first war and sought to prepare their eastern frontier for a further conflict. They did so by fortifying the border (which annoyed the Persians, as the treaty also specified that this shouldn't happen) and by encouraging Persian client states to defect to the Roman side. This is where Armenia comes in. Clients are very useful for powerful states as they provide a way for them to exert their influence without being directly involved. In the mountainous region of the Transcaucasus, it was also easier for the great powers to get their clients to do the fighting for them. Sometimes this indirect control was converted into formal governance from the imperial capital, but both methods of exerting dominance had its advantages (as can be seen in the history of other empires as well). By 530 the two great powers had essentially already been fighting an undeclared war through their clients (which ranged from Arabs in the south to the Huns in the steppes), so I think it's quite understandable why this would escalate into a full-blown war. Peace was sought by both sides by 531 though, since there was a new Persian shah, Khusro I, and he had to first deal with his older brother to secure his hold on the throne. The Romans likewise had just suffered a major defeat and may have had plans to attack the Western Mediterranean, so it made sense for them to agree to a peace as well.

This peace lasted less than a decade, as Khusro I launched sudden invasion of Roman territories in 540. There were many reasons for this, including entreaties from Goths in Italy, who were facing defeat at the hand of the Romans and thus sent an embassy to the east to ask for Persian help. More likely though it was due to general Roman weakness in the area, as emperor Justinian had devoted his efforts to his western reconquests and weakened his eastern frontier. Khusro was an able ruler and centralised the Sassanian administration, so he must have been aware of this. As it was also a tradition for Persian rulers to wage military campaigns to demonstrate their strength, Khusro, having defeated his internal enemies, no doubt saw a chance to strike at the Roman Near East to demonstrate just that. After the initial battles though both sides sought to limit the conflict to the Caucasian mountains - the Syrian and Mesopotamian fortresses were difficult to take and local truces were signed on those fronts once both powers realised that the war there had become a war of attrition. The war however dragged on in the north until 562 as the two powers sought to gain advantages over the other by fighting with and for their local clients, but peace was eventually concluded, with much of the status quo restored.

Then came Justin II, who decided to back Armenian rebels (against Persia) out of Christian solidarity in 572, provoking a war that lasted until 591. Fighting was again mostly restricted to Armenia/the Caucasus, so only a few major gains were made by either side. Stalemate however was broken when a Persian general overthrew Khusro II, who fled to emperor Maurice for protection. Khusro II was restored with Maurice's help, and in return Maurice received the bulk of Armenia and the restoration of important border forts. Then followed a period of co-operation, since Maurice and Khusro were on friendly terms with each other, demonstrating how useful friendly relations can be for both empires. Even before this, the two empires had developed elaborate diplomatic procedures to aid negotiations and both had recognised each other's importance (somewhat reluctantly) - their enemies were still barbarians, but at least they were not as bad as those uncivilised nomads! This is best seen in the words of a seventh-century historian, who described Rome and Persia as the "two eyes of the earth", a striking metaphor that demonstrates quite clearly the similarities between the two. Wars were fought, yes, but we shouldn't forget that trade, cultural exchange and high-level diplomacy happened constantly as well.

This period of peace was ended by the execution of Maurice by the the usurper Phocas in 602, as it provided an excuse for the ambitious Khusro to expand his empire. Unlike before, Khusro made massive territorial gains and perhaps even thought it possible to fully consolidate them into his empire. It became realistic for the Persians to actually destroy the Roman Empire, an unprecedented opportunity provided by the Roman civil war of 608-10. Allegedly Khusro even rejected several generous peace overtures from the Romans, including one that effectively made it a vassal-state of the Persians. However, this was not to be, thanks to emperor Heraclius' miraculous victories in 626-8, but it was probably the only war between the Romans and the Persians that even had the possibility of ending one empire. The cost was however ruinously high, as both empires had exhausted themselves and left them open for the armies of Islam. In the end, the borders were restored to that of Maurice's time, no doubt because of how difficult it would have been for the Romans to demand more than that - further gains were simply unrealistic after 26 years of unrelenting war.

Common interest in controlling Mesopotamia and east-west trade routes. Persians sought control of the Persian Gulf and had memories of the old Persian Empire's reach to Egypt. The Eastern Mediterranean was a very wealthy region and the Romans refused to budge, but there wasn't much impetus for either side to completely overwhelm the other. The Roman Empire was quite large, and the Persian Empire while not as large had a great deal of internal conflict and invasions from the East.

Because both sides were equally powerful and shared a border far from either one's heartland it is understandable that there would be frequent warfare

Basically, Armenia was a buffer state between both the Roman and Persian Empires. As such, both Rome and Persia wished to have influence there over the other. Armenia was weak enough compared to her more powerful neighbors made her ideal to have influence over, and often was controlled to varying degrees over time by varying empires and regimes, but yet distant enough from the centers of power, and defensible from attack due to the rugged terrain, enough so that conquest and integration into the various empires in the region was problematic. Armenian leadership over time was often successful in playing her more powerful neighbors off of one another.