OK - this is something I've been steeped in for a while, let's see if I can answer this appropriately. I'm happy to edit and update this comment as appropriate if specific sources are needed.

Route 66 was established in 1926 along with a number of other numbered highways during the creation of the US Highway System. Because it connected Chicago and Los Angeles, it quickly became a major thoroughfare for folks migrating west. For example, during the Dust Bowl in the 30s, many Oklahomans pulled up stakes and used 66 to head to California (The book Grapes of Wrath is a great example for this; though it's a fictional tale, the situation is based in reality.)

Will Rogers, arguably the most famous entertainer/personality of the late 20s and early 30s, traveled from Oklahoma to California to achieve his fame. When he died in a plane crash in 1935, the road was unofficially renamed the Will Rogers Highway (a sign proclaiming this is even featured in the film adaptation of Grapes of Wrath). Much later, in 1952, the road was officially designated with the same name in a big tourism push to great fanfare.

When the United States entered World War II, US 66 was fully paved (something many highways had yet to achieve). As such, troop convoys out west utilized the highway for moving military equipment.

After the war, tourism by automobile was mainstream. California remained a popular destination (as did natural landmarks like the Grand Canyon, not far off the route in Arizona) and 66 had already been established as a reliable path to the west. Businesses thrived off the tourist dollar; Route 66 was known as the 'Main Street of America' as it was often Main Street of the many small towns it traveled through. Roadside attractions drew curious travelers, diners were plentiful, and motels were packed. (It was also the beneficiary of national tourism awareness thanks to the U.S. Highway 66 Association, which was incorporated until 1976.)

When the Interstates were built, much of that traffic diverted and a lot of small towns died. The highway was federally de-certified in 1985, but the formation of historic state associations in later years (along with nostalgia tourism) has kept it alive.

Today, many Route 66 towns are seeing an increase in tourism, especially from international visitors (I'm based in Tulsa, OK and meet traveling groups of Aussies and New Zealand folks several times a year.) The highway is essentially a big outdoor museum that showcases what America was like in the mid-20th Century and provides a great variety of North American landscape. Some of it has been updated, some of it is abandoned, but all of it tells a story. I've traveled Route 66 from end-to-end and work with several state historic associations if you have any specific questions.

Some good resources for this are:

Route 66: The Mother Road by Michael Wallis (1990) -- it captures what Route 66 was like just as revival was stirring. The author also weaves in the overall story of the road to help illustrate the importance of preserving the highway for future generations.

Father of Route 66: The Story of Cy Avery by Susan Croce Kelly -- this talks about Cyrus Avery, the main architect behind the establishment of Route 66 and the greater US Highway System.

The Illustrated Route 66 Historical Atlas by Jim Hinckley -- this one is a bit more focused on modern tourism destinations of the road, but also paints a great state-by-state picture of the history of the road and how it's changed over the years.

edit: clarified some statements, added info about Will Rogers

On a related note, I remember learning in school the US highway system was mainly built to help in moving troops in case of invasion (this would be under Eisenhower). Is this true and did the same logic apply to old Route 66 and other intercontinental trails?