The first comic with Batman showed him using guns, much like the writers' inspiration The Shadow, but they stopped showing up before anything that makes him distinct, such as the dead parents and Robin. The first recurring villain, however, shows up before the last "normal" gun use, and was killed off (chemicals to the face followed by a fall) before being brought back the next month. Also, as far as I can tell, the only things he killed with a gun were a pair of vampires (who came back, albeit much later) and a monster. These tell us several things: first, the fact that the main antecedent of Batman used guns can tell us that they were pretty common in comics up until that point. Second, their being written out early shows that the writers didn't like them for a particular reason. We can also see that the use of guns didn't actually impede the use of recurring villains.

From all this, my interpretation is that Finger and Kane took the guns out due to what they wanted the character to be. Even if occasionally using potentially lethal force, a vigilante is much less morally grey if he's not setting out with an exclusively lethal weapon than if he is. It makes the killings seem less about the defense of self or others and much more an extralegal killing. For The Shadow, a gun was okay because he was more about plumbing society's depths and was in a much more lurid series, but Finger and Kane wanted more of a hero than an antihero. Additionally, taking out guns played up fisticuffs, which helped place Batman much more firmly in the rising Superhero genre and attendant tastes, which was handy since he ran around in spandex and a mask.