No, precisely the opposite. Slavery was expanding, and slave property was the most valuable non-land source of capital in the United States. The predominant political issue of the 1850s and the election of 1860 was the expansion of slavery into the territories - something that would not have been an issue had slavery been on the decline.