They knew an invasion was coming, they just didn't know when and they didn't know where. It would have been impossible for Nazi Germany to miss the buildup of material in Great Britain. Ships, tanks, trucks, planes, everything needed to be supplied in great numbers and it would have been impossible to hide it all. The Allies did their best to intercept reconnaissance flights but not all those efforts were successful.
The best thing the Allies could do was actively encourage the German belief that the invasion was coming but deceive them as to where. To this end, Operation Bodyguard was established, the deception plan intended to protect the size, location and time of the invasion. Vast amounts of resources were allocated to Bodyguard with quite a few Nazi spies utilised as double agents to feed misinformation to the Nazi intelligence service. The main goal of Bodyguard was to deceive the Nazi's into thinking that the invasion would come at Calais or perhaps even Norway. Calais was the obvious choice for the invasion, it was the closest point to Great Britain and it possessed the port facilities that Nazi authorities were convinced the Allies would need to supply their invasion.
To fuel this belief, Bodyguard established phantom armies in the area around Dover. These armies, such as the First US Army Group or FUSAG, were created using fake radio traffic, inflatable, planes, tanks and trucks and fake newspaper articles and were used to convince the Nazi's that Calais was indeed the target of the invasion. That Patton was placed in command of the FUSAG lent credence to this deception as Nazi commanders were sure Patton would be participating in the invasion. When the invasion did come, Nazi intelligence knew Patton was still on England which helped convince them that the Normandy attack was a feint and that the real invasion would still come at Calais, preventing the movement of reinforcements to the Normandy region.