From what I gathered from Ken Burns' Vietnam War documentary series, and some point Robert McNamara tried to collect as much quantifiable data the US Army could create during the war and analyze if it could create any valuable information.
According to the documentary, the attempt was not successful because the sheer amount of data that was being collected could not be analyzed in an efficient manner. What fascinates me is that Robert McNamara was basically looking for a modern computer to process the information before computers had the capability to do so.
So has anyone actually taken the data and analyzed it using modern computational methods? Is the data even available nowadays? If it is available, what possible information could be extracted that could be valuable to historians today?
So has anyone actually taken the data and analyzed it using modern computational methods? Is the data even available nowadays? If it is available, what possible information could be extracted that could be valuable to historians today?
So the big key is that a lot of this data isn't available, and depending on the classification level of the data, if it has been analyzed it either isn't public knowledge what has been found (or even if it has been analyzed). Remember: unclassified data can be aggregated to a level that is classified.
That said, some data over the years has been released.
For instance, the US Air Force has released THOR: Theater History of Operations Reports for download which includes:
These records cover airpower records from a variety of sources, theaters, and with various caveats as listed on the website.
Some of this data has been analyzed by people. There are various visualizations of where bombs were dropped, for instance, that you can find online.
And I should point out that it wasn't just a matter of analyzing the data - a lot of this data had to be painstakingly put together and digitized by hand due to the nature of converting paper records to digital datasets:
In 2006, Lt Col Jenns Robertson and his team in the Pentagon faced a daunting task. Every week, the Air Force’s Chief of Staff and other senior military officers would ask for the latest on the air war in Iraq and Afghanistan-how many aircraft had flown that week, which ground units they supported, and what munitions they had dropped.
Working in the Air Force’s Operations Directorate, Robertson had access to a wide array of classified data sources, yet the weekly report was tedious to produce. -- Data was not easily searched and often contained only half the picture, forcing Robertson’s team to assemble the report manually every week over the course of several days. He knew there was an easier way
In his spare time, Robertson began creating the Theater History of Operations Reports (THOR), initially a simple Excel spreadsheet that eventually matured into the largest compilation of releasable U.S. air operations data in existence. Robertson tested his database with his team, asking them to generate the Chief’s weekly report twice -- once manually, and again using THOR. The result was impressive: THOR cut the report work from three days to just under an hour.
After receiving Department of Defense approvals, Robertson was able to post THOR files online so interested public citizens could search the data for their own purposes. Robertson understands the vast potential benefits of opening such a large compilation of previously inaccessible data to the public. “I can’t envision all the ways this can be used”.
As quoted, data DOES still get used by the military (as a side note, from personal experience in this field, just about everything is logged) today in current conflicts, but it appears that not much after Vietnam has been released.
I should also point out that there are definitely those who have had access to data for years... and not always from internal to the US. For instance, this Air Power Studies Centre paper on Precision Guided Munitions is from Australia and quotes US military data on the accuracy of weapons from WW2 to the Persian Gulf War. Of note, this piece was written all the way back in 1995, so data sharing between the US government and other governments has been on-going even if that data isn't released to the general public at large.