I'll let George H.W. Bush explain his reasoning himself. In an article that appeared (and subsequently disappeared) in Time Magazine on March 2, 1998, George H.W. Bush and former National Security Advisor Brent Scowcroft penned their reason under the title, "Why We Didn't Remove Saddam". The full text can be found here. (The article is suspiciously missing from Time's own database, which is another story in and of itself.)
Among the highlights of Bush's own reasoning against overthrowing Saddam:
EDIT: I should also mention that the UN mandate calling for the use of force only included expelling Iraqi forces from Kuwait, not an invasion of Iraq itself. So technically, further action would have been illegal under international law. Here's a link to that UN Security Council Resolution.
EDIT 2: a few words for clarity's sake.
You might be interested in Gen. Schwarzkopf's hour-long debriefing from the end of the military campaign, on Youtube. At about 41 minutes they briefly address the same area of questioning as your post.