At which point in history did the word go from a normal phrase to a total taboo in American culture?
That is a difficult question which might not have any easy answers. Randall Kennedy in Nigger: The Strange Career of a Troublesome Word has difficulty tracing the origins of the word, tracing the first known proto-reference to 1619, but notes that the word and its variants remained in use right up until the current day - although not always with the exact same connotation. Or as he put it:
There was often a generational difference in evidence in competing uses of the N-word with the younger people experimenting with nonderogatory versions. On the other hand, while some of my younger relatives are adamantly opposed to any use of nigger, believing it to be only and unalterably a debasing slur, some of my older relatives anticipated by many years the transformation of nigger (or "nigga") that is now widely attributed to the hip-hop culture. Long before the raper Ice-T insisted upon being called a nigger, my father declared that he was proud to be a "stone nigger"--by which he meant a black man without pretensions who was unafraid to enjoy himself openly and loudly despite the objections of condescending whites or insecure blacks.
Certainly, through the 1960s "nigger" was a colloquial term. The Civil Rights movement in the United States definitely opened up awareness of and changing attitude toward the use of the N-word in the public sphere; Martin Luther King, Jr.'s Letter from Birmingham Jail makes it clear:
[...] when your first name becomes "nigger," your middle name becomes "boy" (however old you are) and your last name becomes "John," and your wife and mother are never given the respected title "Mrs."; [...]
This was also seen in works like anti-war film No Vietnamese Ever Called Me Nigger (1968), based on a quote by Mohammed Ali against the draft, showcased the increasing political ramifications of the term.
Books, movies, and products bearing the N-word generally appear less often from the late 1960s-onwards - but it has never completely disappeared, and continues to see occasional use in titles, not to mention contemporary literature and speech. Agatha Christie's 1939 novel was published as Ten Little Niggers in the UK, but the name was changed to And Then There Were None when it came to the US a few weeks later; the book was printed with its original title in Britain through 1977, and in Australia in 1980.
Although at this point the N-word has almost universally become considered offensive and not acceptable in public speech, this was the result of a long road of social pressure and raising awareness, and as a result, the word has gained significant power as a taboo - which can be seen in its use in comedy, such as Blazing Saddles (1974) and the infamous Saturday Night Live Word Association (1975) sketch between Chevy Chase and Richard Pryor; as well the use of "nigga" and related terms in hip-hop...much more in that direction and we hit the 20 year rule cap.