I have struggled with this question quite a bit. Often this question pops up with regards to history and other social sciences due to the nature of their material, versus many STEM fields. While at an elementary/middle school level there is a lot of rote memorization expected (What year was the Declaration of Independance signed?), once you get past that you realize that that's perhaps the least important aspect of history-- especially in the days where you can Google any specifics you're unsure of. Instead, the important aspects of a book to remember for history are things like main points, different interpretations, supporting facts, etc.

Note taking is also tricky because a lot of it comes down to personal preference; however that doesn't mean there aren't better or worse ways of going about it. I actually answer a similar question in this comment, and discuss some of the ways we've traditionally been taught to take notes that are meh, and better ways to actively remember (i.e. active recall). It's a fairly short write-up and it includes some other source material, so definitely check that out.

There are also posts floating around here that discuss technics for information retention that can also be found in books like The Well-Educated Mind by Susan Bauer, and How to Read a Book by Mortimer Adler. These methods usually advocate for at least two readings of a book, where you first go through and look at the index, chapter titles, maybe skim some of the book ahead of time, and read things like introductory and conclusion paragraphs. You then do a full rerad through without stopping-- no note taking, no looking anything up that isn't crucial to your read through. Then you started back at the front, and read through diving into it more, taking notes only on important bits, asking a lot of questions, and looking up any information you're not certain of.

Personally? I hate this method. This is totally personal opinion, and I encourage you to try it out to see how it works for you. But since it's popularly advocated I'd like to share my experience with it here. There is a strong advocacy in these methods that books don't need to be read linearly, and that doing so often harms our comprehension. I have tried this method several times and it just doesn't feel right to me. As someone who really enjoys the act of reading, it felt too disambiguated, too clinical. An analogy that just came to mind is a medical doctor. There are two doctors-- one who treats patients in a clinic, and the other who works in the morgue, going through and doing an autopsy and removing organs. The latter may attain a better understanding of the inner-workings of the patient, but they don't see them as a living, breathing being-- they only see a dead corpse. When I've tried this method I feel like I'm rifling through a corpse; I'm no longer reading, I'm just dissecting and looking to analytically understand. So for me, I prefer to be the first doctor treating the patient as a whole. Hopefully that analogy wasn't too grotesque :D

I would suggest trying different methods out (again check out my initial post and see if anything speaks to you) and see what helps you the most. One thing I'd have you ask yourself is "are my notes useful?" If you're someone who enjoys looking back over your notes, then that's a good sign. If you're someone who will take extensive notes and then never look at them again, that is a sign they're not useful to you. Currently, I'm finding I prefer to get those little sticky tabs that I can use to (sparingly) flag parts of a book I thought were interesting, or quotes that were good, or just important passages. I read physical books and keep all of them, and I find when I do this I tend to have a good spatial memory for being able to track something down later without taking external notes. Again that's just a personal example, and you'll want to find what works best for you and your goals.

Hope this was helpful! It's a subject I'm very interested in as well, let me know if you want to chat about it more!