As far as raw numbers go, the Church Committee from the 1970s (formed after Watergate) still has about the best estimated numbers you can get:

• 500,000 files on individuals and domestic groups

• 130,000 letters opened and photographed from 1940 to 1966

It's hard to get more exact than that due to the level of destruction of materials.

The numbers do seem impossible at first glance if every one of those involved an extreme level of scrutiny, but that was reserved for special cases. Just "having a file" is not quite the same as "being under surveillance". For example, Ayn Rand had a file, starting in 1947. The FBI Director J. Edgar Hoover wanted a compilation of people who testified for the House Un-American Activities Commission and she was noted as friendly; the file starts with a review of The Fountainhead stored for posterity. The file included a 1961 letter upset at Rand's atheism and "odious philosophy" of objectivism, with a response from Hoover himself that pointed out she was most definitely not a Communist.

The file also includes a letter of Rand wanting to talk to Hoover directly, and, curiously, a copy of the book For the New Intellectual that someone had sent anonymously (probably never read).

On the other end of the spectrum you had the surveillance (and much worse) of Martin Luther King Jr. He first had raised the ire of Hoover via an 1959 article in The Nation where he wrote about the lack of black Federal agents, and Hoover from that time essentially disliked MLK personally. In 1962 the Southern Christian Leadership Conference was first investigated starting in 1962 by the Domestic Intelligence Division, handled by the Internal Security Division, and even though the Atlanta office reported there was no significant Communist influence, Hoover pushed back and insisted that King be listed on the infamous "section A" of the reserve index. (Those to be "held" in the case of a national emergency.) In response to a follow-up memo a year later:

This memo reminds me vividly of those I received when Castro took over Cuba. You contended then that Castro and his cohorts were not Communists and not influenced by Communists. Time alone proved you wrong ...

An electronic surveillance program was run from 1963 to 1966. The FBI used (according to Congress's Findings on MLK Assassination) "little discretion" in deciding what to record, even though the original directive was to look for contacts with Communists. The effort became more about unearthing personal information, and a letter using some of the material was sent to King with a copy of one of the tapes:

King, there is only one thing left for you to do. You know what it is. You have just 34 days in which to do (this exact number has been selected for a specific reason, it, has definite practical significance). You are done. There is but one way out for you. You better take it before your filthy fraudulent self is bared to the Nation.

An entirely separate campaign was started in 1967 by COINTELPRO against both black groups and MLK (a year before the MLK assassination). COINTELPRO (Counter Intelligence Program), which ran from 1956-1971, is sometimes confused for FBI surveillance as a whole -- notice the number of years MLK was surveilled otherwise -- and tended to be targeted at groups deemed "subversive" like the Black Panthers and the American Indian Movement; of course as part of that they would also target leaders, so MLK was doubly included. 1967 also saw the start of the GIP, the Ghetto Informant Program, which pried information from people who were in communities in order to gather information. These included people with incidental contact, like "bankers, telephone company employees, and landlords".

In the years things really were out of control -- let's say from 1967 to 1972 -- the "everyone turning their neighbor in" wasn't quite accurate, but if you were in a "suspected radical" community -- black people, hippies, minorities in general -- and were in a large enough city, it is likely there was an FBI informant somewhere. The best estimate is about 7,000 informants for the GIP program at some point. (To be fair, things were utterly off the rails in the United States amongst both leftists and rightists. There were forty-one bombings and arson cases on colleges in just the fall of 1968.)

And what about Hemingway, where the question came from? Well, most of his FBI file is in regarding his days when he worked for for US intelligence on behalf of the American Embassy while he was in Cuba. (This was after he already introduced someone from the FBI as a "member of the Gestapo", so that was considered more a joke than a strike against him.) He assembled a team under the guise of "scientific investigations concerning the migration of Marlin on behalf of the Museum of Natural History, New York City" but the FBI was less than impressed with his prowess and eventually Hemingway and the FBI parted ways in 1944, although Hemingway still sent reports to the Navy in regards to submarines. The occasionally report comes in his pile after, like about the time a journalist from New Zealand challenged him to a duel in 1954 ...

In the letter HEMINGWAY advised Dr. SANCHEZ PESSINO that he had no intention of fighting a duel with Mr. SCOTT, giving as his reasons the fact that he was in ill health and "has a lot of writing to do." HEMINGWAY further stated that he felt sure that a court of honor would not consider this cowardice on his part.

... and one from 1959 which quotes from an interview where he mentioned supporting the Revolutionary Government and hoped the Cubans would regard him as another Cuban, kissing a Cuban flag as a gesture. This was a year before he has the anecdote recounted at the linked article (where he is paranoid about being followed, and has his vehicle bugged) but in all likelihood this had more to do with his own mental state; the file does not include any further notes on his activities until he ended up in a hospital, and later additions to the file come after he died. Hoover himself said later he did not think Hemingway was a Communist, and the level of effort required -- multiple agents -- like they were willing to do with MLK -- would have been more clearly noted in the file.

Hemingway would be on a "watch list" but not have a dedicated agent at all, but just a file that would get added to when his name came up, whereas someone like MLK and his related groups would involve most of the Atlanta branch of the FBI. You ask what it would be like from the other side. In general, extreme measures like the letter to King were not the norm; instead people often just wouldn't know, which is in a way more terrifying. Groups split accusing each other of being FBI informants without knowing who the informants really were. The president of University of California, Clark Kerr, angered Hoover in 1959 when he refused to condemn professors who didn't sign a loyalty oath; as consequence, his career was attacked, and when Johnson wanted to appoint him Secretary of Health it was surreptitiously shot down. Kerr thought he was just unlucky. He only found out he had a file, and what really happened, in 2002.

...

Gentry, C. (2001). J. Edgar Hoover. United Kingdom: Norton.

Brown, J., Lipton, B. C. D. and Morisy, M. (Eds.) Writers Under Surveillance: The FBI Files. (2018). MIT Press.