The only "secret" part of Germany's re-armament was the German airforce school in Russian Lipetsk. Apart from that, Hitler's breach of the Versailles Treaty in terms of the extent of the re-armament was well known to the Allies by means of regular espionage; this knowledge was simply never "formalised" through the diplomatic channels.

In short: Germany did not re-arm in secret. That's a popular myth to sugar-coat the total failure of the appeasement policy towards Hitler.

Sources:

Völker: Die Deutsche Luftwaffe 1933-1939

Deist: Militär, Staat und Gesellschaft

Countries will use the full range of intelligence disciplines including human intelligence (HUMINT), image intelligence (IMINT), signals intelligence (SIGINT) etc. During the Cold War NATO would often see a new Soviet weapon system for the first time in a military parade and frequently overestimated the production of weapons, particularly ballistic missiles.

With regards to Nazi Germany, their rearmament was something of an open secret although probably not the extent of it. There was a policy of appeasement towards Hitler by the rest of Europe during the 1930s and little appetite for militarily enforcing the terms of Versailles.