It really depended on what you were trying to achieve. Obviously the bigger the gun, the greater weight of shell available for firing. This meant that when trying to overcome defences, bigger shells caused more destruction making bigger guns the favourable choice. But maybe the enemy doesn't have any defences, maybe he's the one attacking. In that case a smaller, more manoeuvrable gun would be a better choice, trading weight of shell for the ability to quickly relocate your guns to where they are needed. Smaller guns are also faster to reload, making them more useful against manoeuvrable formations.

The rail guns you speak of weren't very practical either. While capable of immense destructive power, they were difficult to aim with any degree of accuracy which is why they were used against large targets such as at the Siege of Sevastopol. They also required an appropriate rail line which often needed to be constructed to place the gun where you needed it.

Source: Firepower by Bidwell and Graham