I have been reading about this battle, and am amazed that Crassus thought infantry could withstand so many professional horse, especially the archers. The Roman infantry were sitting ducks vs. some of the best horse archers in the world. The Parthians mitigated limited arrows with a vast camel supply of them, and the Parthian heavy horsemen were able to take effect whenever the Roman infantry might fortify too much vs. the missiles. It boggles me that the Romans would have had no inkling of this type of fighting. Crassus was not a 0% general, as he had defeated Spartacus. Perhaps there is no good explanation for such folly, but I wanted to ask none-the-less.
Crassus' army did not have the tools to face the Parthians because the Romans had never before encountered such an enemy. They had never experienced the cavalry and archery-centric tactics of the Parthians, wherein direct confrontation was avoided on all but the most favourable terms. Previous Roman campaigns in the East, though not so far east as Crassus, had met with resounding and almost effortless success against the armies of Pontus and Armenia (Goldsworthy 64). Of course, Pontic and Armenian armies tended to contain large numbers of infantry and fight in a Hellenistic style, with a core of heavy infantry supported by light infantry and cavalry. This was the bread-and-butter of the Romans, and never having encountered the non-committal charge-and-retreat, mass-archery cavalry tactics of the Parthians I think it's not quite fair to expect the Romans to have seen the result of Carrhae coming.
As a result of the lack of real information (though Plutarch suggests Crassus was given these details, even if this were true it is no substitute for actual experience: see Crassus 18), the Roman army composition at Carrhae was, as you say, quite unsuitable for facing the Parthians in the open. Larger numbers of appropriate trained and equipped cavalry and missile troops were required for success in this sort of battle. Unfortunately, while Crassus appears to have done his best with what he had, ordering a hearty counterattack led by his son with a strong force of cavalry, this was defeated and with the death of his son and the reverses suffered by the army, the morale of the Romans and their general appears to have collapsed (Plut. Crassus 25-7). That the Romans were able to withdraw at all from such a situation is impressive enough and testament to their organisation and cohesion under extreme pressure, but the intelligence and army composition flaws that resulted in the tactical disasters at Carrhae also had a fatal psychological effect on the army and its commander. With this collapse in morale, further effective action by the army and its general was all-but impossible, hence the army's disastrous fate after their retreat from the battle (65).
After this battle however, it was the turn of the Parthians to be overconfident when facing Roman armies. Plutarch recounts the rejoicing and arrogance of the Parthians after Carrhae in some detail:
"While he was receiving his applause, Sillaces stood at the door of the banqueting-hall, and after a low obeisance, cast the head of Crassus into the centre of the company. The Parthians lifted it up with clapping of hands and shouts of joy, and at the king's bidding his servants gave Sillaces a seat at the banquet. Then Jason handed his costume of Pentheus to one of the chorus, seized the head of Crassus, and assuming the role of the frenzied Agave, sang these verses through as if inspired:"
"We bring from the mountain A tendril fresh-cut to the palace, A wonderful prey."
"This delighted everybody; but when the following dialogue with the chorus was chanted:"
"Who slew him?"
"Mine is the honour,"
"Pomaxathres, who happened to be one of the banqueters, sprang up and laid hold of the head, feeling that it was more appropriate for him to say this than for Jason. The king was delighted, and bestowed on Pomaxathres the customary gifts, while to Jason he gave a talent. With such a farce as this the expedition of Crassus is said to have closed, just like a tragedy."
"However, worthy punishment overtook both Hyrodes for his cruelty and Surena for his treachery. For not long after this Hyrodes became jealous of the reputation of Surena, and put him to death; and after Hyrodes had lost his son Pacorus, who was defeated in battle by the Romans, and had fallen into a disease which resulted in dropsy, his son Phraates plotted against his life and gave him aconite. And when the disease absorbed the poison so that it was thrown off with it and the patient thereby relieved, Phraates took the shortest path and strangled his father." (Crassus 33.2-5).
Aside from the divine punishment visited upon the Parthians responsible for the defeat of the Romans (Surena) and their humiliation in a mock triumph over the body parts of Crassus (Hyrodes), the Parthians suffered amply for their tactical errors in further encounters with the Romans. Further pitched battles against the Romans led first by Cassius in 51 BC and later by Publius Ventidius in 39 and 38 BC** saw the Parthians suffer major defeats against better prepared Roman leaders and armies, who utilised terrain, cavalry and missile troops more effectively (65). Indeed, the Parthian army was on the whole a distant competitor to the Roman army when the Romans had the appropriate troops. As Goldsworthy puts it:
"If a balanced army kept together and made no mistakes, even moving in battle formation, then a Parthian force could not hope to beat it even in the open. Its horse-archers would have been unable to weaken the enemy significantly enough for the heavy cavalry to have charged successfully. As a result Parthian armies tended to shadow Roman forces watching for an error to exploit, such as the detaching of a small force.* When they made the mistake of actually attacking, they lost." (67).
The campaigns of Corbulo c. 63 BC showed the weakness of the Parthian army against a strong, well-led Roman army. However, the logistical challenges of operating so far east were a huge challenge for the Romans, with distance, disease, the climate and the large numbers of troops required for a successful campaign severely limited Roman progress in the area (68).
Essentially, the Romans had all the answers to the tactical questions posed by the Parthians. But before Crassus' impetuous campaign, typically Republican Roman in its poor intelligence preparation and aggressive movement, the Romans hadn't even encountered the problems they had to solve. Once they had encountered the problems, they solved them. As a result of Parthian military weakness relative to the Romans, it was only with the rise of the much stronger Sassanid Empire in AD 224 that the Iranians became a serious military threat to Rome.
*For instance, Crassus' desperate counter-attack at Carrhae.
**Ventidius won three significant victories against the Parthians at the Cilician Gates and the Amanus Pass in the same year, and decisively at Mount Gindarus the next year. Cassius Dio 49.19-20 gives an account of the latter battle.
Source:
Plutarch, Life of Crassus (Loeb translation).
Goldsworthy, The Roman Army at War, 100 BC- AD 200, 1996, 60-8.