When I was doing my MA, I took notes by hand (this helps with retention, and also I was old school like that). I'd write down main ideas, trying for only one phrase/sentence per paragraph, unless it was something I found particularly interesting. I enjoyed drawing pictures/cartoons to illustrate concepts -- this again helps with retention (particularly if it's a hard concept to grasp) and breaks up the monotony a bit :)
If I had a personal copy of something (e.g. printed out journal article), I would highlight and write down quick paraphrases in the margins. More cartoons, as well, because why not? There is a ton of research that shows that highlighting/underlining on its own does little to no good.
When I moved to the PhD, I switched to typing my notes and my reading was more targeted. So I would skip or skim entire sections of books or articles, and take really detailed notes on other sections. I used Microsoft OneNote, which is an entirely useless programme except for this one use, as far as I can tell.
A few things I did to make my life easier:
Always put the full citation of the work, in the form I needed it in the footnotes, at the top of the page.
Always add a new piece to my bibliography as soon as I started reading it
Added my own thoughts in italics in the margins -- when I went to add things to my thesis, I could often copy things word for word
Make it very clear when I was copying a quote
Keep accurate note of page numbers as I went along, to avoid having to scramble to find these at the last minute
So in short, yes, I highly recommend taking notes. It takes longer and you won't get through as much, but you'll retain much more information over all, in my experience.
On the chance you're a student i'll tell you what I do, as a student, since we might share situations we find ourselves in when reading. I will just about always take notes. How I take them will depend on what I am going to use the book for. My most common way of taking notes is underlining key phrases and then jotting done notes on the side of the page for when I will need to talk in class or if I will need to use it for an unknown paper in the future for the class. I know some people hate writing in book and certainly there are times when it's not an option. If I am reading for a paper I will keep something to write on next to me. I will jot down thoughts, paper section(paragraphs or parts of), or take quotes. I'll do some MLAish junk and just do something like (Art and War pg12) taking just enough of the title to remember the book allowing myself to go back later if I need it and properly cite. If you end up having to do a lot of reading on public transit which I do keep a narrow piece of paper and jot down page numbers. If it's a book I can write in i'll do a little mark by the section I want to go back to. When I get to school or home I'll hopefully go back and look at these sections and do what I need with them. What I have found useful in taking some form of notes is the time it saves when I have to go back and use the readings for papers. This is true for older books used for classes or books I bought for my own interest. The use of the piece of paper with page numbers which I can do for each book has been especially useful time saving when looking at old books. I am changing over and trying to use that method in conjunction with any of way I am taking notes. If you are just reading to enjoy but are coming and asking this question then something has pushed you towards a desire to at least entertain the thought of doing notes.( I love when I can just sit down and simply get lost reading without any worry of taking notes) What might then be a real reason for taking notes is the ability to flush your thoughts out that you are having from your reading. If you read a page and you notice your mind racing off with the information stop and let yourself go. When you are done, or you realize you have gone off on a mind tangent, come back and jot down what you were thinking. What you are doing here is an excellent way to increase your reading comprehension and will help solidify that information, depending on the type of learner you are, to your long term memory. I also find that when I have finished the book I feel more confident in my understanding of what the author was trying to get across not just remembering specific phrases that can get regurgitated(I’m thinking of that bar scene from Good Will Hunting). I hope something here is useful for you.
These days I highlight key passages whenever I read something non-fiction (obviously only if I own the book though :)). If I consider the contents important, I make more or less detailed notes on my laptop.
These are always detailed enough so that I can quote the important parts from the notes only, so there are the book/publication details, page numbers, and the like.