The short answer is that being an "empire" has relatively little to do with geographic size. While there are people who might consider the US an "empire", there is nobody who would refer to the geographically larger Canada as an "empire".
It is difficult to even discuss the concept of "empire" in the context of most of Western history without using Rome as a reference point, but I'm going to start by trying to do that then circle back to Rome as a reference point.
The general characteristics of an Empire are (not every Empire hits every criteria):
There is a reason that these criteria seem like they fit really well with the Roman Empire. The reason is that the entire concept of "Empire" came from the Roman Empire. There were similar situations in the ancient world, like the "King of Kings" in Persia, but the conception of Empire that came down through Western history and was later expanded to be inclusive of East Asian empires is the Roman conception.
During the early Middle Ages the two Christian entities that referred to themselves as empires, the Byzantine Empire and the Holy Roman Empire, had, respectively, really good and fairly plausible claims to be continuations of the Roman Empire.
During the later Middle Ages claims of connection to the Roman Empire became gradually less plausible, from the Russian claim of being the "Third Rome" to various Spanish and French claims of inheriting the Byzantine claims by buying out the claims of some refugee relative of a former Byzantine Emperor (Same guy sold the claim to both I think).
By the early modern period these claims of inheriting imperial status from Rome became less and less important, but the conception of Empire relating to 1. Metropole/colonies 2. Dominant culture or ethnicity 3. Ruled by an Emperor all tended to stick. This conception allowed British, Chinese, and Japanese to fit within the framework even without any purported connection to the Roman Empire.
In summary, empire isn't about geographic size, it's more about how different societies relate to each-other within one state, and to a lesser extent how that state is ruled.