There is no inherent distinction between any of these words - they are interchangeable (although I feel "epoch" is used more in geology or paleontology than history).
There's one main purpose for dividing history into periods, or ages, or eras, which is to make it easier to refer to a specific range of years without having to give them every time. That's literally it. The date range is usually significant for some reason to the person labeling the era or choosing that particular era, and as a result we get many overlapping labels. For instance:
The Georgian era/period refers to the reigns of Georges I through IV of the United Kingdom, 1714-1830. The term is only relevant to British history and isn't used to refer to other places at that time.
The Regency refers to the sub-period 1811-1820, when the future George IV ruled as regent for George III.
However, it is also used more broadly (sometimes explicitly as the long regency) to refer to roughly 1800-1830 because there's a continuity between these years, both politically and in terms of art/fashion/decorative arts/style which is useful to refer to as a bloc. You see this applied regardless of geography, sometimes.
Meanwhile, in American history, people refer to the entire period between initial European invasion/settlement ca. 1600 and the Revolution as the colonial era. (Other colonized countries also have their own colonial eras depending on their specific histories.) You will also find "colonial" used to mean specifically the lead-up to the Revolution, which is wrong but common.
And also the Federalist era, 1788-1801, when the Federalist Party was in power in the nascent US - a political identifier much like "Georgian".
But federalist also gets applied to the "long regency" period in the US when it comes to fashion, probably because there's no better term. (That's the handy thing about royalty, they make it very easy to refer to longer date ranges.) Past Patterns, for instance, dates their Federalist Era Patterns page as 1789-1829.
The Era of Good Feelings is a rarely used term for 1815-1825 in American history, but it does exist. It refers to a supposed time of optimism and national unity before tensions between the north and south began to increase in the antebellum period (1830s to the Civil War).
The Romantic period, to go in another direction, is defined by an artistic movement and has no defined beginning and end - it starts in the late eighteenth century and ends in the mid-nineteenth.
And I can't believe I almost forgot my favorite, the long eighteenth century. This is similarly undefined, and similarly more universal. People use it to refer to the late seventeenth century through the early nineteenth century, to recognize that history doesn't actually change on the '00 and to help address subjects that mainly relate to the eighteenth century but overlap with the previous or next one.
This isn't even getting into the relevant Continental periods! As you can see, which period you use depends entirely on what you're doing. If you're writing about family law in New York in the 1790s, you're probably going to use "Federal". If you're writing about the disapproval of the aristocracy among the British middle classes from the 1780s through 1820s, "late Georgian" makes the most sense. And so on.