This is my answer about how armies fought in the time. I've written about this nuMero us times but I can only find that from my phone. It's a presentist idea that this was inefficient or silly, it was the most effective means of war. Further, rifles were not common because of how expensive they were, how long it took to reload, and how they were difficult to maintain.

In terms of 19th century warfare I recommend this further reading by myself:

• We often hear about how war changed in the 20th century due to machine guns and tanks, but how was warfare revolutionized in the 19th century through technological advances? Were there any precursors to the 20th century's changes?

• How did the Napoleonic style of war (formations, cavalry charges, volley fire) die out?

The tl;dr: of this all though is that:

1. Organization was key. First and foremost the strict formations were used to maintain order -- that's what line tactics were all about -- order. Being able to keep everyone together and deliver orders clearly.

2. By maintaining order in the approach you keep everyone together in a cohesive unit. This provides protection in numbers and mass from enemy cavalry. More simply if you all sprint up to the enemy and then try to form up there they are not only shooting at you in an organized fashion while you're trying to reform but they are also likely to attach bayonets and break you while you're already disorganized or send a cavalry charge right into your disorganized mass. This era of warfare is first and foremost one of morale, not killing the most people as possible. It's about staying on the field while your enemy runs first -- everything was built around this. Mass volleys, lots of smoke, bayonet charges, artillery batteries, etc.

3. Not that many would die in the approach regardless. These weapons were incredibly inaccurate at the time and its only at a relatively close range that casualties would start to mount and by that point you were firing back.

Though with that said it's not some homogenous mass. Frederick the Great was famous for using rapid marches, firing 2-3 quick volleys and then immediately going in for a decisive bayonet charge instead of the slow march and maneuver game. Really as we move on in the 18th century we see a break from slow, maneuver and methodical along with doctrinal concepts like fortress warfare into what is considered Napoleonic Warfare where lightning fast attacks, high operational tempo, living off the land rather than slow methodical sieges and supply chains, etc. would replace it. So there was certainly change as the conditions changed; they were not dumb!

For your specific musings:

try and come in from behind/flank

They certainly tried to do this but it was generally reserved for cavalry for obvious reasons.

hide behind trees, lie down, etc.

Light infantry would also do this but:

1. Lying down was impossible with muzzle loaders and would only be applicable to really 1866 onward when breech loaders became a "thing"

2. Light infantry was generally not effective or used in any serious fashion until the Napoleonic Wars. The weapons were simply not accurate enough and the doctrine of the time was more about slow, methodical maneuver based warfare and siege warfare which was not really necessary to include a large skirmishing body in.

Quick short answer - I am sure you will get more detailed answers from some of the real experts on this sub shortly. It was very difficult to maneuver large formations of soldiers. Moving a regiment (1000 -2500 men) from a column to a line formation was slow, time consuming, done at the speed of a slow walk/march. Now increase that to a lot of regiments and you have a serious problem just keeping everyone from not running into each other. Keeping a large number of men under control required a high level of discipline - thus 'lying down' would not work because control would be lost, plus it is really hard to load a muzzleloader while prone. Armies tried to use heavy cavalry to achieve surprise and attack flanks, rear etc. The use of cavalry during the US Civil war was generally limited to recon efforts, thus the units lined up and blasted each other. Europeans tried to use cavalry for shock effect. Hope one of the experts on this sub chimes in on this.