So, as I said above, the Viking Society for Northern Research's An Introduction to Old Norse (3 volumes, but Vol. 1 is all you need for the language, really) is a far better source for learning about the language than Byock's book.
Otherwise, get yourself a copy of The Sagas of Icelanders (ed. Jane Smiley), and basically everything Penguin or the Folio Society has published in translation.
As far as history and society goes, look at William Ian Miller's Bloodtaking and Peacemaking, Stefan Brink and Neil Price's The Viking World, and Else Rosedahl's The Vikings. Andrew MacDonald and Angus Somerville have also recently put out an anthology of greatest hits from the viking age called The Vikings and their Age, and since they were my undergrad advisers, I figure I should plug them, too.
Some recommendations:
Jesse Byock, Viking Language 1 (2013). If you're serious about studying Icelandic history and the sagas, Old Norse is a must. This is an excellent primer accessible to pretty much anyone with a diligent desire to learn the language. Most importantly, Byock is an eminent scholar, and he peppers the text with bits and pieces of the critical history of the Viking Age. Even if you don't end up completing the course of instruction, paging through this book offers a great quick and easy introduction to the history you're asking for.
Nancy Marie Brown, The Far Traveler (2008). This is a profoundly engaging read about one woman's quest to learn more about the saga heroine Gudrid. It's not a comprehensive history of the sagas, but Brown walks the reader through her own fascinating research. You'll walk away with a good idea about all the different ways we can learn about the sagas: learning the language, reading the texts, excavating the sites, and sailing the seas.
And there's no reason not to jump right into the sources. I've found the Penguin Sagas of the Icelanders to be a page-turner. Njal's Saga is another favorite about feuding farmers. Or if you're more interested in some of the far-out magic and swords and dragons, The Saga of the Volsungs is the place to go. Again, Penguin editions might be best: they're cheap, they're readable (unlike some older translations free online), and they've usually got terrific introductions. (Never pass up a good introduction!)
Happy reading!