Well, very few were able to build secret rooms. They would have to have access to a non-Jew's home (as Jewish homes were confiscated) and that non-Jew would have to be sympathetic and allow them to build such a thing, as well as providing food, etc. - not easy when food was rationed. If someone noticed you took extra food, you could be turned in.

However, it was clear that the situation was going downhill quite early. For example, Otto Frank took his family out of Germany in plenty of time. So they would've had time to build things - given the opportunity.

Nazis didn't always find them because most homes didn't have secret rooms. You don't look for rabbits in your house, because there is no reason to expect there is a rabbit in your house - unless you see rabbit poop. The tipoff would be a wall that doesn't line up, for example, or the aforementioned informer who notices your household goes through bread at twice the normal rate.