When you're calm, your heart goes ba-boom.....ba-boom. When you're not, it goes more like boombabombaboombababoombaboom. Modern psychology would say that emotions are - very broadly - a combination of bodily expression and the mental interpretation of bodily expression. One of the bodily expressions of emotion, particularly to do with calm vs tension, is in the speed of the heartbeat (which is regulated by the autonomic nervous system and which brain areas like the medulla play a role in).
In light of this, it's perhaps unsurprising that a 1996 paper by Eric Jager argues that 'from antiquity until after the Middle Ages the heart was the traditional seat of emotion' - it's a pretty obvious connection to make given that everyone feels their heart speed up and slow down when they feel different emotions. So, for example, Gilgamesh tames Enkidu in the Epic Of Gilgamesh poems "not by means of arms but with the power of the heart over the mind", according to Ole M. Høystad in A History Of The Heart. Gilgamesh's mother asks of the creator, in the Epic, '"Why have you imposed--nay, inflicted!--a restless heart on my son, Gilgamesh!"
Ancient Greek philosophers like Aristotle, in De Anima, argued that the heart is not only the source of emotion but of thought as well (there has historically been much debate about the relationship between thought and emotion, and the extent to which emotion is cognitive). However, for the physician Galen, writing a few hundred years later, it was necessary to point out that the esteemed author Aristotle was wrong; the brain was the source of perception and cognition, and the source of emotion in some ways is also the liver, which Galen thought generated the bodily 'humours' that give us modern words for temperament like 'sanguine', 'phlegmatic' and 'melancholic', and which he thought played a role in temperament.
Similarly, in the New Testament, a passage from Mark (7:21) sees the heart as the source of more than just emotion, but more cognitive thoughts too: "For it is from within, out of a person’s heart, that evil thoughts come—sexual immorality, theft, murder, adultery, greed, malice, deceit, lewdness, envy, slander, arrogance and folly. All these evils come from inside and defile a person."
From some of these disparate sources, a tradition develops in the writing of Augustine of Hippo, an important Christian author of the 4th century who discusses his heart (cor meum) in his book Confessions several dozen of times. For Augustine, the heart is "the indivisible, authentic centre of human life": "my heart is where I am, such as I am". For Augustine, the heart is distinctly verbal, and has ears and a voice, and which has words inscribed upon it; the heart is his conscience; Augustine sees the heart and memory as being intimately linked. By the High Middle Ages, Augustine's verbal heart has turned into a heart that is described as being like a scroll, or a tablet; the heart can be written upon. And so, in medieval legends, it's often said that when a particularly blessed/miraculous person died, that when their heart was opened up, they would find - in the case of Ignatius of Antioch - 'Jesus Christus' written in gold inside his heart. Jager goes on to argue that the idea of the 'inner book' died very slowly, even as medical knowledge about the role of the heart vs the brain in behaviour advanced; Puritans in New England, for example, discussed the writing of the heart.
Alongside this tradition, the 12th century writer Bernardus Silvestris specifically identifies the brain as the place of thought and intellect, while the heart is "the animating spark of the body, nurse of its life, the creative principle and harmonizing bond of the senses, the central link in the human structure...mainstay of our nature, king, governor, creator."
Of course, this tradition became effectively metaphorical after William Harvey's research on the heart, which in 1616 showed that the heart was essentially just a pump, and not quite the seat of emotions the way that had previously been thought. One result of this is that the dualism of Descartes (who was 20 years old when Harvey presented his paper on the heart) accordingly doesn't identify the soul with the heart in any way. Instead, Descartes' soul - and the experience of emotions - occurs entirely in a spiritual plane divorced from material things such as blood-pumping hearts.
However, Høystad in A History Of The Heart argues that while Descartes and Harvey ended the reign of the heart in one way, they had little effect on the way the average person talked about the heart; instead, Høystad argues that Montaigne and particularly Shakespeare had a massive impact on the modern conception of the heart in every day culture. Shakespeare's plays brim with references to the heart; the famous 'to be or not to be' speech mentions 'heartache', while later in the same scene Claudius suggests sending Hamlet abroad:
"Haply the seas, and countries different,
With variable objects, shall expel
This something-settled matter in his heart,
Whereon his brains still beating puts him thus
From fashion of himself."
And certainly Shakespeare's conception of the heart as contrasted with the mind or the brain; looks much like the one in modern pop music than the Harvey/Cartesian conception of the heart as a pump, in titles like the Motown track 'This Old Heart Of Mine (Is Weak For You)', most famously sung by the Isley Brothers, or the Bacharach-David track 'Anyone Who Had A Heart' (most famously sung by Dionne Warwick or Dusty Springfield) or Blondie's 'Heart Of Glass'. Or perhaps you could argue that Billy Ray Cyrus's 'Achy Breaky Heart', in which Cyrus personifies his heart as a 'he' whose reaction to being told of the end of a relationship would be quite explosive, is in some ways quite similar to the verbal heart of Augustine of Hippo.