Amusement parks have been dated as far back as the 1500s. Bakken opened in 1583 when large crowds gathered around a newly found natural spring (pure water was a rare thing). The large crowds attracted venders and entertainers, who set up shop near the stream.
Of course, an amusement park, like Bakken, Tivoli Gardens, and Coney Island, are much different than a theme park. An amusement park has many themes, varying widely from attraction to attraction. A theme park attempts continuity with its storyline and keeps either a single theme or several themes in demarcated areas.
While many claim Disneyland as the original theme park, I would have to disagree and give the title to Knott's Berry Farm. Walter Knott, after incredible success with his chicken dinner restaurant, set up Old Western themed attractions (including stores, buildings, and a haunted town) to entertain guests. He wanted to put guests in a setting where they could envision themselves in the Old West and enjoy related activities. Walt Disney, a friend of Knott's, visited Knott's Berry Farm many times for inspiration for his own California Theme park. It could be argued, however, that since Knott did not charge admission until 1968 that Walt Disney created the first enclosed theme park in America.
Further Reading:
Walter Knott: Twentieth Century pioneer by Norman E. Nygaard
Variations on a Theme Park: the New American City and the End of Public Space by Michael Sorkin
Magic Lands: Wester Cityscapes and American Culture after 1940 by John Findlay
Are We There Yet?: the Golden Age of American Family Vacations by Susan Rugh