"Books" in ancient texts tend to be divisions rather a lot shorter than modern "volumes," such that frequently modern editions contain several books in one volume. What we translate as "books" is Latin libri or Greek αἱ βύβλοι, which describe not only completed textual works but the individual rolls of papyrus on which they were written. These rolls could only be so long before becoming absurd to manipulate. By the Hellenistic Period several standard scroll sizes were generally manufactured, which are themselves based on norms from the Classical Period and earlier. These scrolls were generally large enough to fit a decently long text--say, a single speech of Cicero--but were not so enormous as to fit an entire volume of the Cambridge Ancient History. More like a chapter, or several chapters than a volume. Besides division according to the pragmatic requirement of space, the division of texts into books often coincided with thematic changes in the text, meaning that a book might actually be quite short if its thematic material lent itself to brevity. The Iliad, for example, nearly 16000 lines in 24 books; the Odyssey contains around 12000 lines in the same number of books. These divisions, which did not exist in the original text, were decided largely by the Alexandrian scholars, who divided the text of Homer up into thematic chunks based on the action. The Aeneid, on the other hand, contains fewer lines, slightly under 10000, but is divided into only 12 books. Prose books tended to be much denser than verse, simply because with line breaks fewer verse lines can fit into the same amount of space. But the 40 books of Polybius or the 78 books of Plutarch's Moralia (many of them really very short) are not especially absurd.

Livy's another story, Livy's a bit ridiculous. But consider the circumstances of Livy's work. Livy's history was quite literally his life's work, he wrote it for over forty years, publishing it in little sections as he went--his death in 17 AD clearly left the work unfinished, as Book 142 ends rather abruptly with the death of the younger Drusus some eight years before Livy's death. We know relatively little about Livy. He was a Paduan, and therefore not as "Roman" as, say, Pollio. He was (naturally) well-educated, and had some connections by marriage to rhetorical circles. Tacitus remarks that he knew Augustus and was friends with him, despite the pro-Pompeian leanings he expressed to him (and which are a clear undercurrent throughout the surviving books of the work). He does not, however, quite unlike his contemporaries Horace and Pollio, appear ever to have even dabbled in political or military life. There is no tradition of it, Livy gives no indication of it himself (and, given his penchant for describing totally implausible military actions with troops teleporting and shit it's hard to see how he could have had any military experience at all), and he is not known through any other text to have participated in politics or the civil war. Instead, Livy appears to have dedicated pretty much every waking moment of his life from the end of the civil war to his death to writing this thing, which is just about how long it would take to write out his full text, if what we have is any indication. Livy's rather an exception because of the circumstances of his work, however: other ancient authors are not so prolific.

A "book" (liber) in this context is closer to the word "chapter", and specifically refers to scrolls. So, for example, the Aeneid is made up of twelve books, each one taking up about 20-30 pages or so in a modern translation and printing format. By contrast, in the Penguin edition of Livy, each book takes up about 50-100 pages.

So the broad answer is that it is not quite as monumental as the modern term would suggest, but these are still massive undertakings. We only have about a quarter of Livy's work but even that takes up significant shelf space.

As for why they are so long, I have a copy of Anthony Beevor's Second World War that runs to about eight hundred pages, despite only covering six years.