It wasn't always like that, no. It's origin is from Charlemagne and was carried by his Carolingian line through the 9th century. The Carolingian, and if a Medievalist reads this correct me if I'm wrong, Merovingian previously, concept of transfer of power would see the sons of a ruler divide his lands equally and form new 'kinglets' as they were called. The result was a constantly devolving authority which ended in the latter 9th century with Charles the Fat and his death, and subsequent collapse of the Carolingian dynasty. However, Pope Leo had granted Charlemagne the title of 'Emperor' through coronation and translatio imperii (transfer of rule). The end result was the formation of the political entity known as the Holy Roman Empire.

It wouldn't be called the 'Holy Roman Empire' until the 13th century, but that's how it was 'born' if you will. So to recap simply: Charlemagne was made Emperor of Rome, and because of the way that princes inherited land under the Carolingian system many German states were founded, though never consolidated like they would eventually in France. I'm not a Medievalist, I admit, but that's how I understand that it happened based on my modest exposure to Medieval history in grad school.