He didn't do it purposefully. Plutarch (and I'm using the open-source translation here because I don't have my copy handy) writes that 'when the enemy endeavored to cut off [Caesar's] communication by sea, he was forced to divert that danger by setting fire to his own ships, which, after burning the docks, thence spread on and destroyed the great library' - i.e., Caesar had no intention of burning the library, the fire merely spread out of his control.