Indeed, in its usage in the 18th and early 19th centuries, the term was used in much the same way as you mentioned - a neutral or even positive descriptor (another example might be "Spaniard.") It was frequently given in the early 19th century by immigration officials who either received no adequate response to their inquiries for the immigrants' names, or simply could understand what they were told - and as such would frequently list them as chinaman/chinawoman, or Chinese. At times ethnically Chinese would even self-refer as "Chinamen," as in a letter from Norman Asing, a Chinese restaurant owner and community leader in San Francisco to the then governor of California, John Bigler in 1852 regarding the proposed immigration restrictions. Asing wrote:

To His Excellency Gov. Bigler

Sir: I am a Chinaman, a republican, and a lover of free institutions; am much attached to the principles of the government of the United States, and therefore take the liberty of addressing you as the chief of the government of this State.

So, when did that semantic shift - from Chinaman being a neutral or even positive descriptor and even self-applied demonym - to it becoming a derogatory, racist term? There are, of course, several factors - some internal to the East Asian experience in the US, and some involving China itself. We'll look at both, in turn.

In terms of the term's usage in America and Europe, we can see - to no one's great surprise - that Euro-Americans were not terribly picky about which Asian people to whom they applied the term chinaman. Chinese, sure, but also Japanese, Korean, Vietnamese. If the "face was yellow", they were equally Chinamen in the eyes of the majority of Americans and Europeans. This is well demonstrated by the "nickname" for the future Japanese Grand Admiral, and subsequently dubbed in the West as the "Nelson of the East," Togo Heihachiro (東郷 平八郎) given by his British shipmates during his training in Britain, who called him "Johnny Chinaman." Given the even by then longstanding national antipathy between Qing China and Meiji Japan, it's unsurprising that such a "nickname" led the young Togo, on more than one occasion to "put an end to it by blows." Likewise, the obviously flippant and over-broad label swiftly came to be regarded by the Chinese as a dismissal by the white man of their unique culture and nationality, since it was obvious that they meant "all East Asians" when they said "Chinamen" ... in marked contrast to when they referred to Frenchmen or Dutchmen.

One of the cementing factors over the course of the 19th century leading to the term becoming derogatory rather than neutral, though, was the attitude by Americans as a whole regarding Chinese and East Asian in general immigration. The so-called "Yellow Peril" era - a phrase coined, by the by, in 1895 by the German Kaiser Wilhelm II - that codified the already rising fear among Europeans and Americans both that "The Orientals" were dangerous and potentially destabilizing to their world order. That fear was easily given legs in 1905 by the victory of the Japanese over the Russian Tsar in the Ruso-Japanese War - seemingly proving that "The Orientals" posed a significant military threat to the Western Powers and must be appropriate dealt with. That, of course, was not the beginning of the Yellow Peril, only the birth of the term itself. The actual sentiment, as we saw in Asing's letter to the Governor of California, went back decades earlier. To quote again from Asing's 1852 open letter:

You are deeply convinced you say “that to enhance the prosperity and preserve the tranquility of this State, Asiatic immigration must be checked.” This, your Excellency, is but one step towards a retrograde movement of the government, [...] You argue that this is a republic of a particular race—that the Constitution of the United States admits of no asylum to any other than the pale face. This proposition is false in the extreme, and you know it.

[...] we would beg to remind you that when your nation was a wilderness, and the nation from which you sprung barbarous, we exercised most of the arts and virtues of civilized life; that we are possessed of a language and a literature, and that men skilled in science and the arts are numerous among us; that the productions of our manufactories, our sail, and workshops, form no small share of the commerce of the world; [...] And we beg to remark, that so far as the history of our race in California goes, it stamps with the test of truth the fact that we are not the degraded race you would make us. [...] As far as regards the color and complexion of our race, we are perfectly aware that our population have been a little more tan than yours.

[...] as far as the aristocracy of skin is concerned, ours might compare with many of the European races [...]. I am a naturalized citizen, your Excellency, of Charleston, South Carolina, and a Christian, too; and so hope you will stand corrected in your assertion “that none of the Asiatic class” as you are pleased to term them, have applied for benefits under our naturalization act. I could point out to you numbers of citizens, all over the whole continent, who have taken advantage of your hospitality and citizenship, and I defy you to say that our race have ever abused that hospitality or forfeited their claim on this [government] by an infringement on the laws [...]You find us peculiarly peaceable and orderly.

In spite of such eloquent defense of the Chinese' rights in the US, the remainder of the 19th century proved out the power of xenophobia and racism over rhetoric. In a particularly virulent piece in the early 1870's - leading up to the Chinese Massacre of 1871 in Los Angeles - editor of the New York Tribune and famed orator of the era Horace Greely intoned, "The Chinese are uncivilized, unclean, and filthy beyond all conception, without any of the higher domestic or social relations; lustful and sensual in their dispositions; every female is a prostitute of the basest order." In the face of such xenophobia, the Chinese Exclusion Act was passed and enacted in 1882, and would remain on the books and at least partially enforced until 1943, when it was finally repealed (though it was superceded by the just-as-bad Immigration Act of 1924, which outright banned immigration from all Asian countries until 1965 when it was struck down and replaced by the Immigration and Nationality Act.)

The overthrow of the Qing Dynasty and the establishment of the Republic of China in 1912 likely also set in motion a further drive to "do away with" the old exonyms, much as it did re: Sino-Japanese relations. One of the (relatively minor, but still important) sticking points between China and Japan over the 2nd through 4th decades of the 20th century was what Japan was going to call this new, non-imperial Chinese Republic. Briefly, the old Japanese name for China - 支那 (Shina) - came to be regarded by the RoC by the late 30’s as not just archaic but an outright insult and epithet – by referring to China by its “old” name, that was understood to imply its antiquity and obsolescence. As brought up by /u/CaptainPyjamaShark, we might think of the Japanese insistence on calling China and the Chinese 支那人, "Sina-man" (or as it morphed into by the late 30's and the 2nd Sino-Japanese War, 支那猪 ... "Sina-pig")... as similar to "Chinaman" and probably even more so the term "Celestial" in English - a hearkening back to the period of foreign Qing rule, when the Chinese were Qing Dynasty Slaves (清国奴)... i.e. "Servants of the Great Celestial Qing" (Note: a pejorative exonym, not an endonym, because apparently that needed to be spelled out). One can imagine why the citizens of the Republic of China would not have been so happy about continually being referred to as such in either Japanese or English, especially when it had been so mis-applied and jumped on by whites as yet another epithet. From Chuimei Ho, "the usage quickly became pejorative when readers interpreted it as meaning that Chinese, absurdly in view of their obviously inferior economic and legal status, considered themselves to be superior to ordinary humans. Calling a mere laundryman 'celestial' was an irony that even the least educated anti-Chinese white hoodlum could understand. So it rapidly became an insult for some."

Asing, Norman (1852). "To His Excellency Gov. Bigler," in Daily Alta California.

Clements, Jonathan (2010). Admiral Togo: Nelson of the East.

Falk, Edwin A. (1936). Togo and the Rise of Japanese Sea Power.

Yang, Tim (2013). The Malleable Yet Undying Nature of the Yellow Peril.

Mail Tribune 2011 email interview with Dr. Chuimei Ho.

We're about a good 500 years before Europeans ever directly encountered Chinese, much less started applying epithets to them. Still, consider checking out The History of China Podcast. We're up to the late Tang Dynasty in the late 8th Century, when the imperial court trying to stay above 50 mph in a flaming, smoke-filled bus full of passengers or else the bomb goes off... What do you do, hotshot? What do you do?!

If I may add on something: Celestial is also a commonly used term for Chinese on Deadwood. Is that historically correct, and where did that term come from?

As a person born in Chicago, I have often heard the term "chinaman" to refer to someone in government who can get things done for you with a phone call--i.e. I made a call to my chinaman in the alderman's office. Any insight as to where this usage came from?

Followup question if I may, is there anything historical to indicate why some US cities still openly call areas of their cities "Chinatown", when other areas associated with the names of ethnic groups are no longer used? I grew up and still live in the south and in my work I often used official maps and platts that were part of the public record that used some very offensive names and as new documents are put in the public record, those names have been purged.

Edited for a word and some context.

First of all, the term Chinaman is not compatible with Frenchman/Englishman/Dutchman. The equal usage of the term would be Chineseman (or the other way: Franceman/Englandman/Netherlandsman).

One brief discussion that I'm yet to see discussed here is the influence of author/playwright Frank Chin and his attempts to reclaim the term "Chinaman" (he even wrote a famous play "The Chickencoop Chinaman"), especially after it was well understood to be past its time. Chin's argument is that Chinese Americans that grew up during the early 20th century before the Asian American movement of the late-1960s would have referred to themselves--and been referred to--as Chinamen. Therefore, it wouldn't make sense for them to rebrand themselves as Chinese Americans, since that's not the social/political identity that they grew up with.

Zhongguo means Middle Kingdom or china. Ren means person.

Zhongguo ren literally means chinaman and it's how they refer to themselves.

Edit: Thats not a joke or a racist comment. Chinaperson to be non-binary? ease up reddit.