There's a podcast series called the history of Rome which won some podcast awards. I found it to be a very accessible and easy to understand history without being excessively verbose. I'm not sure how the scholars on this sub feel about it, but at least you can make progress while you drive to work :)

There are many choices to consider here depending on what kind of things you are looking for and I'll name quite a few books here that are both readable and will give you an accurate sense of what modern historians think about the Roman Empire. I also encourage you to think about the empire as an entity that lasted well into the middle ages - I hope you don't mind some of my later recommendations! A brief note about reading primary sources though: they are essential for historians, but I'm not so sure if they are the best introduction to Roman history for a novice. They only ever provide one perspective on the events, whilst modern historians endeavour to look beyond the sources and synthesise all the different accounts into a reasonable interpretation of the past. Primary sources will never tell you exactly what happened. Secondary sources can't do that either, but the narrative they tell is at least a bit closer to what actually happened.

To get a general sense of Roman history, try The Romans: From Village to Empire (2012). If you are interested in military affairs, I recommend Adrian Goldsworthy's The Complete Roman Army (2003) for, well, a complete manual on what the Roman army was all about. His volumes on the Punic Wars, Julius Caesar, Antony and Cleopatra, and Augustus are all excellent too and are all very readable introductions to the complex world of Roman politics. To get a sense of historians' views on the empire in the fourth and fifth centuries, consider Goldsworthy's The Fall of the West (2009) and Peter Heather's The Fall of the Roman Empire (2005). Mike Duncan's History of Rome podcast is another great way to introduce yourself to Roman history and is very entertaining at the same time.

Imperial Rome however lived on after 476! The sequel to Heather's book, The Restoration of Rome (2014), is an excellent overview of what happened after Rome 'fell', which is rather important since the Roman Empire lived on in the east and the post-Roman world in the west was never as bad as what past historians, such as Gibbon, told us. Late antiquity really wasn't as cataclysmic as some people thought it was and this was really down to the amazing work done by Peter Brown, particularly in his The World of Late Antiquity (1971), which dealt with cultural history from the third century to the seventh, so I heartily recommend that as well, as it literally transformed our understanding of the late empire. You might be told that Gibbon is a great way to introduce yourself to the history of the late empire - he is definitely not. Give Brown a try instead, he writes very eloquently and his contribution is valuable even now, to the extent that he should be the name on everyone's lips rather than Gibbon if you want to learn about late Rome/Byzantium. If you are interested in Christianity as well, which cannot be separated from any discussion of the Roman Empire from the fourth century onwards, Brown's The Rise of Western Christendom, 200-1000 (2002) is the book to read. Of course, late antiquity was also an age of violence and material deprivation despite its cultural and political continuity with the earlier Roman period. This is best encapsulated in Bryan Ward-Perkins' The Fall of Rome and the End of Civilization (2006).

For the Byzantine Empire (though it really was just the Eastern Roman Empire, historians just use it to distinguish between two phases of the same empire), consider Norwich's A Short History of Byzantium (1997). Even though it is a bit too old for my taste and only provides a very basic narrative through a surface reading of the sources, it will at least introduce you to the key events of the empire's history. The empire lasted for more than a thousand years, so it is obviously quite difficult to summarise them in a readable way - Norwich in my opinion did the best job he could in the circumstances. For a more modern approach, I heartily recommend Judith Herrin's Byzantium: The Surprising Life of a Medieval Empire (2007), as she examined the empire's history thematically rather than chronologically, and looked at fields often unexplored in popular histories of the Roman world, such as on the role of women, art and the experiences of the average citizen. Averil Cameron (one of the biggest names in Byzantine studies) also wrote The Byzantines (2009) and it is very good as well. It is supposed to be an introductory text aimed at both non-specialists and Byzantinists, but I think Herrin's book is more readable and provides a more innovative approach to studying the empire. Still, give them both a try and hopefully one of these books will get you enthused about everything Byzantine! For more specific treatments of certain periods, Mark Whittow's The Making of Orthodox Byzantium, 600-1025 (1996) and Peter Frankopan's The First Crusade: the Call from the East (2012) are both very accessible and good overviews of the specific periods they cover. The History of Byzantium podcast is great as well, especially as a sequel to Duncan's podcast, so maybe give that a listen too :)

The standard introductory textbook is Boatwright et al Rome: From Village to Empire.

Try The Twelve Caesars by Suetonius. It's small bites, lots of scandalous stuff and it's the basic source for early imperial history. You'll like it. You can always find a copy in second hand book stores, and I'm pretty sure it's a common free e book.

The Rise of Rome by Anthony Everitt is a fantastic read on the subject as well.

I liked The Roman Way by Edith Hamilton.

You can start by searching the sub and read those book lists.

http://www.reddit.com/r/AskHistorians/wiki/books/europe#wiki_ancient_rome

http://www.reddit.com/r/AskHistorians/comments/timi4/the_askhistorians_master_book_list/

Historical fiction: Colleen McCullough's Masters of Rome series takes you from 100 BCE to 27 BCE.

Sources: Livy does a history of the republic, Tacitus' Annales gives you the beginning of the empire, and Suetonius' Lives of the Caesars gives you biographies of the first 12 emperors. Ammianus Marcellinus is much later, but fascinating.

David Potter is a phenomenal scholar; search his works by what interests you. Edward Gibbon's Decline and Fall of the Roman Empire is a bit dated, but phenomenal.

Roman Realities by Finley Hooper is a good general history.

edit: Hooper, not Cooper.

People have already recommended you some excellent primary sources to get you started, so I'm going to recommend you read The Oxford History of the Roman World each section was written by a different specialist on a certain aspect of the Roman world, with subjects ranging from Early Rome and Italy to Roman Historians to Roman Art and Architecture. Hope this helped you!

If your a reader, and you like people, Plutarch's Roman Lives is a great resource on the major participants in Rome's history. It features figures such as the Gracchi brothers, Sulla, Gaius Marius, Cato, Scipio Aemilianus, and more than just Caesar.

Are you interested only in the empire/Empire (note the difference the capital letter makes), either as a construct (Empire), in the practices and realities of imperial rule (empire, e.g. life in Roman Britain), or as Roman history during the 'Empire'? (I use scare quotes only because as a chronological term it's often used to refer to the Principate onwards, whereas Rome already had numerous imperial conquests during its Republic.) Or would you also be looking to find out more about Rome's longer history in general?

Forgive the re-post, but here's what I recommended to a similar post in the r/history thread:

If you can still find a modern copy, go for Michael Grant's classic History of Rome. Holland's more recent Rubicon, as mentioned by others here, covers only the Republic, and although he writes well, he skips a whole lot of the earlier mythologies and histories (e.g. Cato or Livy) to dwell a lot on the later Republic. (His bibliography is actually quite rich with modern references, but his actual text unfortunately doesn't incorporate much of those new perspectives.) Scullard is OK, but if you find Grant somewhat dry, Scullard's text book will be even more desiccated. Great detail, on the other hand. The Fall of the Roman Empire is always going to require a different text altogether. Skip Gibbon (unless you really want to slog through those thousands of pages): his argument is still vital in the scholarship as a kind of first detailed analysis, but falls short in light of more modern propositions, including of late especially the (different) theses by Peter Heather and Bryan Ward-Perkins. Most modern works on the Fall will inevitably assimilate a broader review of other views, anyway. I wouldn't go to Norwich on Byzantium as a follow-up on the 'Roman Empire' after its Fall, personally, simply because he is a bit reductive; but it's true there's otherwise not much popular history on the Byzantine Empire. Geographically speaking, my opinion is that the best continuation narrative is Chris Wickham's gargantuan tome on The Inheritance of Rome: Illuminating the Dark Ages, 400-1000.

There's also another popular history by Simon Baker, but I'm not familiar with it. If you're just interested in the Empire itself, a nice introductory work is Greg Woolf's Rome: An Empire's Story (2013).

I don't think you can separate the visual art of Ancient Rome from its history. So while Duncan's podcast is great, I'd suggest a more visual approach for dipping your toe in the water for the first time.

https://www.khanacademy.org/humanities/ancient-art-civilizations/roman

You can study Ancient Rome for a lifetime in words and never get the kind of picture in your head that first video gives you. So that's where I'd start. After that it's easy reading and half of it is videos and pictures. From there you can branch out to the other suggestions in this thread.

Yes it's mostly about the art history of Ancient Rome, but all around are things that should spark your curiosity. If a particular subject doesn't interest you, it's easy to skip, but at least watch the first video.

Aha, something I can answer well here! Wheee!

I'm not a historian either, but like you have a strong amateur's interest in Rome. Hands down, the best resource I have found have been TTC (The Teaching Company)'s History of Ancient Rome series. They actually hire university professors to deliver their lectures in a studio and film them, so the information quality is subperb! I learned so much from these videos that I have never heard from regular documentary sources.

Bear in mind, the videos are very expensive. Of course, some people who love knowledge but don't have deep wallets choose to pirate these videos, but I don't condone that kind of behaviour.

Spesifically the empire part of Roman history, spanning roughly 50 BCE to 483 AD? Or all of Roman history (700 BCE to 483 AD) from the merit based kingship to the republic and finally to the empire? If you want the ladder the historian Livy is a great resource for a step past an introductory view of Rome, although much of his work spans the Republic aspect of Roman history. (Personally my favorite)