Pulitzer Prize winning The Making of the Atomic Bomb by Richard Rhodes may be the first book you want to read. It won the Pulitzer Prize for General Non-Fiction, the National Book Award for Nonfiction, and a National Book Critics Circle Award.

There are potentially two negatives about the book. First, as a book dealing with the history of science it took greater concentration for me to read the science parts because I lack the scientific background. Obviously, your mileage may vary. Second, it was published over 30 years ago but seems to hold up well.