I agree with Keyilan that the "China" we talk about today included, in its core, the territory of the Xia, Shang, and Zhou. However, I do see the Qin as a disruption of the old order. Let me go about explaining further chronologically:
XIA - probably Erlitou. But nobody really knows. There is no strong evidence for the Xia's existence beyond the imaginations and machinations of Warring States/Qin/Han historians (and this putative existence was always political). The Erlitou culture did exist, among other regional population centers around the time, and the Erlitou seems to fit nicely within the Xia dates - however, this is hardly evidence of kingdom or empire.
SHANG - Very small territory, undoubtedly important, but hardly an empire.
ZHOU - I am not terribly impressed with the Zhou. Although they clearly had some hegemonic power, it was always tenuous, and didn't last all that long. I think it is difficult to really give them "empire" status. First among equals may be the best statement about the Zhou's first 300-ish years, and then only nominally so afterward. By the rise of the Qin, the Zhou was a small player in the Warring States power game.
QIN - this was a major shift in the state politics of the core region of what we today call China. Qin formation and consolidation was entirely based on disrupting the old state system. Central government control was the core of the Qin system. This is why I suggest Qin was the first Empire in the region. It didn't work too well, the attempts at centralization only hardened the old regional powers against the Qin, and once they were able to consolidate regional power, they toppled the Qin. Nonetheless, within 100 years of the Han's existence, the Qin model of centralization would prevail, under the domineering hand of Han Wudi.
The extent of Wudi's Han territory was indeed vast, and similar to the core of China today.
One interesting side note - I was arguing with a Chinese friend about how Tibet was absolutely not part of China historically. He pointed out that the Qing were in control of Tibet militarily from the 18th century onward. I countered with this Han map - and asked him this: Is northern Vietnam and Korea "China"? He was stubborn. Side note over.

I think there is good evidence and reason to suggest that Qin was the first empire. The Xia-Shang-Zhou stuff was always rhetorical, designed to inflate the legitimacy of the newest pretender to power - establish a direct link back through the empires and you have legitimacy.

Considered by whom? At least over here in and around China, the Xia (夏) are considered the first dynasty, preceding the Qin by around 2000 years. Qin has the status that it does because it had the greatest reach of any other dynasty up to that point, and it did so by conquering a number of small regional powers.

It almost seems like China was defined by whoever had the largest mass of it under their control.

Nah. China's got a pretty generally agreed upon definition, at least at the core. China in this sense is not (中國 zhongguo in Chinese), the modern nation that is about 65 years old, or 103 years old if you're referring to Republican China, now Taiwan. China here means the area China (中華 zhonghua in Chinese). So Zhou and Xia were dynasties in the region of China (the latter definition), with Qin being a later dynasty covering a much greater area of the region of China. It's just that, at the time of the Xia or Zhou, there wasn't a hugely significant other state that left significant archaeological or historical evidence. Zhou and Xia and Shang were, through the historical lens, the most significant.