There are a few important things to note going on here. I'm going to assume you're American and your exposure is mostly to the American Jewish community.

First, while Judaism believes that Jewish status passes down matrilineally, not everyone whose mother is Jewish actually identifies as Jewish. A lot of the children and grandchildren of intermarriages do not identify as Jewish. This would be true in societies where having Jewish ancestry wasn't a major social handicap, but being Jewish was (such as, say, late 19th century Western Europe)--there would be a particular pressure to identify more with the non-Jewish side.

Second, intermarriage wasn't all that common. If you're American, the American Jewish rate of intermarriage is something like 50%. This is very much novel. While intermarriage has basically always happened, in the past it was much rarer. The rate in the US was much lower in the early or mid-20th century, and was probably lower still in Eastern Europe. It may've been notable in Western Europe, but I'm unaware of any particular figures for measuring it.

Third, there aren't all that many specifically Jewish names. Except for a name like Cohen (or one of its many derivatives), most names could easily be found among non-Jews. A lot of Yiddish-origin last names are German names too, and were often Germanicized to the German equivalent at some point. A name like "Rosen" strongly suggests Jewish ancestry, but it could just be German. The same is true of other Yiddish/German names. Many Jews took names of colors as their last names (Gelb, Grun, Schwartz, Weiss), but those are common German names too. In Eastern Europe many Jews adopted place names as last names, but I believe those are fairly common among non-Jewish Eastern Europeans too.

Some names suggest Jewish ancestry, like the aforementioned examples, or a name like Gluck (or especially Glick), or a last name that's an Eastern European city with an -off ending instead of an -ov ending, or a patronymic with a name also common to non-Jews. When non-Jews have those names, it's not obvious whether they have Jewish ancestry or whether they happen to have a name that they share with a lot of Jews. A lot of non-Jews could very well have Jewish-origin last names and it wouldn't be obvious. There are some unambiguously Jewish names (Cohen and its variants, a patronymic that is particularly Jewish, etc) but not all that many.

Fourth, a lot of Jews Anglicized their names, sometimes by picking a similar one, sometimes by translating. A name like Greenberg or Grun or Grin might get changed to Green, Braun to Brown (also a common non-Jewish German name)

Fifth, European Jews tended to adopt surnames relatively late. This means there hasn't been so long for names to move back and forth. The Jewish community who had surnames much earlier (Sefardi Jews) don't live in America in large numbers, and many of their names are similarly just Spanish last names perhaps with an unusal spelling.

Sixth, I suspect non-Jews who ended up with Jewish names may've changed them or Anglicized them at a higher rate, especially in the past when discrimination based on names would've been more noticeable. Even if you're not Jewish, you might have trouble getting your kids into a good prep school with a name like Yitzkowitz or something.

But, I'd really like to challenge the premise of the question. There are lots of Jews without distinctly Jewish names. Going through my synagogue directory, there are a significant number that aren't identifiable Jewish names. Of course there are Cohens and Lebowitzes and -off names and Shapiros and Schwartzes and Rosenbergs (and less obviously Jewish names, but still ones that suggest Jewish ancestry), but there are also Smiths and Coopers and Browns and a whole bunch of English surnames. And some of the non-Jewish Shermans or Weisses or Golds (or nearly any German or Eastern European name) that you know could've gotten it from a Jewish paternal ancestor.