In the case of Aktion T-4, the Nazi eugenics program which sought to eliminate disabled and mentally ill people from the German gene pool by means of mass murder, the public was lied to outright. Richard J. Evans writes in his The Third Reich at War:

The families and relatives of the victims were only informed of their transfer to a killing centre after it had taken place. A further letter was sent by the receiving institution registering their safe arrival but warning relatives not to visit them until they had safely settled in. Of course, by the time the relatives received the letter, the patient was in fact already dead. Some time later, the families were notified that the patient had died of a heart attack, pneumonia, tuberculosis, or a similar ailment, from a list provided by the T-4 office and augmented by notes made during their examination on arrival. Aware that they were in some sense acting illegally, the physicians used false names when signing the death certificate, as well as, of course, appending a false date, to make it appear that death had occurred days or weeks after arrival [...].

Even so, doctors throughout the nations knew what was going on and ordinary people started to be suspicious in the light of repeated disappearances and deaths of relatives and acquaintances. In the end, too many people knew and amazingly some even protested openly, like judge Lothar Kreyssig and bishop Clemens August von Galen. So Hitler decided to halt the program, at least for adult German citizens - killing of children continued because it was on a far smaller scale.

There is an interesting film you can watch if you are interested, called Amen by Costa Gavras. While the film itself is not based on a true story, it does a good effort to retain historical accuracy when it comes to these things.

Generally, the killings were gradual. The first to go where the mentally ill and mentally handicapped (as /u/Lele_ describes). Aside from killing they'd also sterilize people. Many Jews, Communists and Romuva were targeted with the excuse that they had genetic deficiencies or that someone in their family had genetic deficiencies, and thus they also had deficient genes (genetic deficiencies included everything from mental illness, disfigurement, chronic illness up to homosexuality, depression, reclusiveness, phobias etc). Special courts were set up that decided whether someone would be sterilized or not. Before 1939, at least 600 000 people were sterilized.

The first Jews and Romuva to be killed en masse weren't Germans, they were Poles. After the invasion of Poland, the Germans quickly established death squads that ran around in Poland killing Jews, Communists and Romuva people. It should be noted that Communism was seen as a Jewish arm, and the Romuva were simply seen as intruders, morally bankrupt and undesirables. Hardly anyone protested about the Gypsies, because nearly everyone had some notion to hate them.

The small-scale killings that were performed in Germany were then quickly applied to Poland and Czechoslovakia (and later France) in a more industrialized scale. At first gassing was done in modified trucks or towed gas chambers. When it was apparent that death squads and mobile gas chambers were not enough to do so much murder, they decided to establish death camps. This was sometime in 1941. Concentration labor camps already existed, were many Communists, Jews, Romuva and others were detained and worked to death, but they were few and rare at that point.

Generally not many people were aware of what was going on after the death camps were established, or they thought it was much more small-scale than in reality. Those who did know, either didn't say anything for fear of what would happen to them, or they thought it was for the good of the state. There were rumors, but they only really concerned those targeted. The rest thought it was propaganda spread by the enemy or just over-exaggeration.

Those who were arrested to be killed were given instructions that they would be deported or they'd be transported to a work camp. The same was told to anyone who would ask on their behalf.