That's his secret. He was actually on a toilet the entire time! The King himself asked him the same question in England: http://www.npr.org/blogs/waitwait/2012/03/02/147812749/on-transatlantic-bathroom-breaks

Lindbergh explained that in his airplane his chair was made of wicker and there was a hole in it. And there was a funnel below that hole. And his waste, whenever nature called, would go down through there into sort of an aluminum can. And so he explained that and said that rather than show up with it in Le Bourget, the airport that he landed in, that he just dropped it over France.

Lindbergh certainly illustrates the example of a long flight however this was a common problem in many long range missions after WWI as long range bombing and reconnaissance evolved, you can see this would be a common issue for all bombers in Europe and the Pacific in WWII.

Although I cannot find a single reference the Heinkel He111 displayed in the Spanish Air Force museum near Madrid feature a flexible pipe next to the pilot side windows with a small cone like mouth, purpose was obvious, it is just an open pipe to the exterior. I can only guess that B-17, B-24, Lancasters, Betty's and so on could only feature similar solutions as the technical options for this would not be very varied.