The main reason is that it was really difficult to sync the recorded sound (dialogue, music and effects) with the moving pictures. Before The Jazz Singer published in -27 there were attempts to use film projector and fonographs or other equipment simultaneously. This didn't work out too well, because the early projectors were rather loud and there was No way to raise the volume of the Sound high enough, so the audience was not able to hear the sound properly. This was especially a problem in bigger theatres. And as I mentioned above, it was difficult to make the sound and film sync in a satisfying way. I have experience in synching translated texts to silent films while they were shown with old projector, and it's quite difficult. Trying to make the sound fit to the moving picture wold be even more challenging and would require constant concentration. The early projectors with film reels were also used manually, so the speed and length of the film could change a little depending on the person operating with the equipment. This added another layer of difficulty to the process of syncing the sound.

During the 20's the suitable technology became available. I'm not an expert when it comes to the exact development in sound technology, since I am a cultural historian, so I'm not qualified to describe how these innovations exactly worked, technology-wise. Anyone is more than welcome to fill this void in my answer.

However, I want to mention the fact that so called Silent Films made before 1927 were rarely silent in the most literal sense. It was very common to play music live during the show. The style and quality of the music was tied to the surrounding. If the films were shown in bigger theatres in cities, there could have been larger orchestras/bands available. However, if the show took place in smaller cities or villages in country side, the music was provided by local musicians with instruments they had. Usually a piano. Sometimes there were sheets with "official" scores composed specifically for particular films. The extent in which they were used is not documented too well. The dialogue was sometimes replaced with a live narration. This method was used especially when the films were foreign and the text cards were written in a foreign language not familiar to the majority of the audience (for example english or german in Finland where I live)

(sorry about the grammar, english is not my first language)