Here's the clip I'm referring to: https://www.youtube.com/watch?v=cg1a5QlCdYo
QI is a good show, but sometimes get their facts wrong. Stephen Fry is a very good host and intelligent man, but he's reading facts from cue-cards and doesn't know everything under the sun.
The columns are, in fact, slightly curved. Modern measurements have confirmed that this is intentional and not a flaw, or the result of the materials or buildings sagging over the centuries.
Vitruvius, the Roman author, architect, and civil engineer from the first century AD, believed that a straight line over a long distance would appear to be sagging, so he reasoned (or assumed) that the architects of Acropolis wanted the lines to look straight, therefore they built it without any straight lines, to create the optical illusion and the impression that all the lines and columns are straight.
I say he assumed because I'm not familiar with any evidence as to the intentions of the builders and designers when they chose to build it that way. It was built, or the construction of the building was overseen, by Pheidias who was a sculptor. He might have had other artistic reasons for the bulging columns and lack of straight lines than wanting everything to look perfectly straight.
James Whitley's book "The Archaeology of Democracy: Classical Athens" mentions the lack of perfectly straight lines and cites modern studies that have measured them.