Based on shooting both types, the main problem I see is recovering your sight picture after working the lever action, which is especially noticeable when prone. Essentially you are disrupting your sight picture upwards and taking the end of the rifle out of your body line when you operate the lever, whereas with a bolt action, particularly one with a turned down bolt handle, the end of the rifle remains in line with your body, allowing you to re-acquire the target faster.

I would be interested if there were army studies of this issue in the 1870s/1880s period, but as I am at work I'm not going to Google anything firearm related.