I always wondered if archaeologists could discover a lot more if they didn't have designated areas for excavation, but rather could dig up anything, anywhere.
For instance, in Romania, we know where some settlements were 2000 years ago, but we can't really dig there because there's a sprawling city on top.
Archaeology is a time-consuming and expensive way to practice history. Archaeology is not a random and hurried effort to find treasure. It is an attempt to extract complex, subtle, and diverse information from a site with a comprehensive, exact method. Because of this, archaeologists attempt to select sites that are most likely to yield information that is appropriate to the interests of the archaeologists and/or of the time.
In addition, there is salvage archaeology - retrieving information from a site that has been accidentally discovered, often through construction or some other land disturbance.
Archaeologists often know about sites, which they "bank" for a time when funds allow for excavation or for when techniques are advanced enough to address questions we cannot imagine being answered (thirty years ago we couldn't imagine the level of DNA retrieval now possible).
For your example: every time there is a fire or a building demolition, freeing a lot for new construction, there is an opportunity for archaeology to be conducted, but it needs to have public support and financing for that approach to work. San Francisco requires just this sort of archaeological work, and the process has yielded amazing results, but it is expensive, making redevelopment costly. Most cities do not require this to happen, so many sites are destroyed for redevelopment.
Archaeology presents complex questions about cost and return. In view of high costs, archaeologists are careful about selecting sites.