Why was the Library at Alexandria burned down?
By accident. After Caesar defeated Pompey at the battle of Pharsalus in 48 BCE, Pompey fled to Alexandria. He himself was murdered by a Ptolemaic officer, but in pursuit of Pompey's forces, Caesar became enmeshed in a conflict over the city. In a domestic civil war between Ptolemy XIII and Cleopatra, he ended up trapped in the city, occupied by superior hostile forces, and there was fighting in the streets. This is the so-called [Alexandrian War of 47 BCE](http://en.wikipedia.org/wiki/Siege_of_Alexandria_(47_BC)).
Various much later sources report that in the course of this fighting, a fire started, possibly as a result of Caesar giving orders to burn the Ptolemaic fleet; that's the version given by Orosius, Historia adversus paganos 6.15.31-2 (5th cent.). Aulus Gellius, Attic nights 7.17.3 (2nd cent.), instead reports that the fire was "accidentally" (forte) started by auxiliary forces. An anonymous account called the Alexandrian war doesn't mention any fire, though Caesar, Civil War 3.111, reports on Caesar successfully burning the Pompeian fleet. Remarkably, the Alexandrian war takes care to point out in its opening paragraph (1.1) that Alexandria was exceptionally fire-proof:
nam <ab> incendio fere tuta est Alexandria, quod sine contignatione ac materia sunt aedificia et structuris ac fornicibus continentur tectaque sunt rudere aut pavimentis.
For Alexandria is practically secure from fire, since the buildings have no floorboards or timber, and they are put together with arched construction and have roofs of stonework or tiling.
In this fire the royal library was burned. Different sources give different figures for exactly how many books were destroyed, and the record is confused how easily scribes could mix up septuaginta milia and septingenta milia (70,000 and 700,000), in Aulus Gellius and Ammianus (22.16.12-13, 4th cent.), and quadraqinta/quadringenta milia, in Seneca (De tranq. animi 9.4-5, 1st cent.) and Orosius. On the face of it the larger figures look more likely: I say this because we know (a) the royal library was the larger collection in the city, (b) we have testimony that puts the size of the library at 200,000 books a decade or two after its founding, and ca. 500,000 books by the mid-3rd century BCE; and (c) there's an anecdote in Plutarch, Antony 58.9, that Antony swiped 200,000 books from Pergamon and gave them to Cleopatra, apparently to make up for the loss a few years earlier.
TL;DR: collateral damage.
And why was it never "rebuilt"?
The public collection, or "outer" library, survived. Testimony from the 2nd cent. onwards explicitly tells us that it was housed at the temple of Serapis on the acropolis; probably that had always been its location (by analogy with the Athenaion at Pergamon, and based on what looks like a reference to a library at a temple in Kallimachos fr. iamb. 191). (One piece of testimony, by Ammianus Marcellinus, is tainted by getting the royal and public libraries thoroughly mixed up, and Orosius bizarrely denies that the secondary collection ever existed. They're both wrong.)
The outer library was probably always the place to be, anyway. That seems to be the point of the names "royal" and "outer". When ancient writers mention the Mouseion (lit. "temple of the Muse(s)"), it's the public library they're talking about. By the standards of its time this library was certainly an important research collection; just not a colossal monster like the royal library had been. The Ptolemaic dynasty had been responsible for building up the royal collection, at great effort and expense, and once they fell, that impetus faded.
Alexandria continued to be an important centre of intellectual activity for a few centuries, and the 2nd-cent. writer Strabo (Geography 17.8.1) tells us that the Roman emperor continued to appoint the priest in charge of the Mouseion; but as the centuries wore on, it withered away in much the same way as libraries anywhere else in the Greco-Roman world. No further particular acts of destruction need to be posited (though there are some that people like to point to, usually as a pretext for bashing the religion of their choice; that line of reasoning never seems to get extended to other major libraries of antiquity, for some reason). If only a tiny cross-section of books survived from the Palatine library in Rome, or the Athenaion in Pergamon, or many other important libraries, there's no need to go looking for special explanations for Alexandria. The passing of time was more than capable of destroying books, all by itself.