I've been watching Ken Burns' documentary and they depict the soldiers fighting in formation standing close together. This seems like a dumb idea considering the gatling gun and the "shotgun" style projectiles they used in canons. Wouldn't the military leaders realize this and adopt better strategies?
Well, this is a case of hindsight being 20/20 in some ways. The prevailing thought within military circles in the mid-19th century was largely based on tactics developed by Napoleon just a few decades before. Although Napoleon certainly didn't invent the double-rank battle line, he did use it effectively to take full advantage of slow-loading rifle fire. The idea was that you group your soldiers in tight formation ("rows" like you mentioned above) so that they could (1) be more easily commanded by a central directive for their unique movement/action (2) resist cavalry charges by virtue of the tight formation, which a horse is not likely to engage, and (3) present massed, concentrated fire on a single point. The rows also served the purpose of allowing a steady stream of fire in large sheets, which came with terrific regularity if the troops were well drilled and practiced at firing, kneeling, reloading, and firing again.
The generals who would plan, lead, and fight the ACW (American Civil War) were largely students of the military academies of the day, and read works like William Hardee's 'Rifle and Light Infantry Tactics' and Winfield Scott's 'Infantry Tactics, etc.' These texts were inspired by Napoleon's tactics and also proclaimed that in order to take a position or move off an enemy, you massed your forces, assaulted with artillery, concentrated fire when in musket range, and then give them the bayonet.
This can certainly be an effective strategy when dealing with an inferior force, a poorly trained one, or one surprised/on its heels. As you mentioned, however, if attacking a well-trained enemy with artillery support, this can also be a suicidal proposition. For this reason, many generals quickly came to the conclusion that outright fighting in a situation where an enemy of equal strength faced you was something to avoid if at all possible. Whenever possible, generals tried to flank their opponents or "steal a march" on them to cut them off or take an advantageous position (high ground, for example). Although Grant and Lee ran into each other in classic "Civil War era" battle fashion a number of times during Grant's '64 campaign from Wilderness to Petersburg, if you read up on the campaign, you'll see that Grant was always trying to turn Lee's flank, and until Petersburg, kept JUST missing out on accomplishing that. The same could be said of Sherman in Georgia, on his way to Atlanta, where he kept trying to outmaneuver Johnson, but kept getting outmaneuvered himself (until he finally did work his way into Atlanta).
There were plenty of cases where generals got their armies in formation, and attacked fortified, prepared, well trained troops (Lee at Gettysburg, McClellan at Antietam, Grant at Cold Harbor, Hood at Franklin) but pretty much every general with a solid head on his shoulders realized by 1862 that maneuver, not sheer force or will, was the key to victory.
[Sources: Shelby Foote, 'The Civil War: A Narrative, vol 1 & 2'; James McPherson, 'Battle Cry of Freedom'; Jay Winik, 'April, 1865'; Bruce Catton, 'Mr. Lincoln's Army' & 'Never Call Retreat']