Was it just gigantic highly pressurized oxygen tanks? The ones they have on commercial airlines now don't last that long, even for the pilots. Did they have some kind of oxygen generating thing that was far more dense?
They used oxygen tanks. Boeing B-17s and Consolidated B-24s (for example) used bright yellow G-1 oxygen tanks with a volume of 1000 cubic inches but a relatively low internal pressure of 400psi. They took up a lot of space but didn't weigh very much. There's a photo of some of them here:
https://www.flickr.com/photos/charlestilford/3885900648
And also lower down the page here:
http://warbirdsnews.com/warbirds-news/aussie-b-24m-liberator-restoration-update-april-2016.html
According to the pilot's manual, a typical B-17 had eighteen G-1 tanks. Each tank could supply one crewmember with oxygen for five hours at 30,000 feet, plus there were a number of F-1 cylinders for the turret crew, plus there were emergency oxygen cylinders, raising the total man-hours of the oxygen supply to 22 hours at 30k:
https://books.google.co.uk/books?id=BTJxDwAAQBAJ&printsec=frontcover&source=gbs_ge_summary_r&cad=0#v=onepage&q&f=false
Mission duration from the UK to Berlin and back was eight hours or more at that altitude but a surplus was required in case of emergencies, and because it was a war there were a lot of emergencies. Modern commercial airliners have oxygen tanks in the hold - back in 2008 Quantas had trouble when a tank in one of their 747s blew up, punching a hole in the side of the airliner:
https://www.nytimes.com/2008/08/30/world/asia/30qantas.html
The masks-dropping-from-ceiling oxygen system uses a separate, chemical oxygen generation system, and is only intended to supply oxygen until the pilots can descend to below 10,000 feet. The pilots themselves have separate oxygen bottles, and there are also portable units in case of emergency:
http://www.askcaptainlim.com/-emergencies-flying-50/1186-using-the-emergency-oxygen-bottle-in-flight.html
I think the key thing is that oxygen bottles take up space but don't weigh a great deal, and furthermore they can be mounted in locations where they don't eat into crew space - in the B-17 above they're mounted in the ceiling, and in modern airliners they're in the walls of the hold, where they're likely to do less damage if one of the tanks ruptures.