It's from his first state of the union, and he's expressely saying that while revolution is justified if the constitution is being negated, it is not justified by policies as regard slavery (at this time he's not talking about emancipation but policies on fugitive slaves).

It's best to read the whole thing but I'll pick out a couple of bits;

in 1787, one of the declared objects for ordaining and establishing the Constitution was "to form a more perfect Union." But if destruction of the Union by one or by a part only of the States be lawfully possible, the Union is less perfect than before the Constitution, having lost the vital element of perpetuity.

It follows from these views that no State upon its own mere motion can lawfully get out of the Union; that resolves and ordinances to that effect are legally void, and that acts of violence within any State or States against the authority of the United States are insurrectionary or revolutionary, according to circumstances.

All profess to be content in the Union if all constitutional rights can be maintained. Is it true, then, that any right plainly written in the Constitution has been denied?

He rejects secession specifically here;

For instance, why may not any portion of a new confederacy a year or two hence arbitrarily secede again, precisely as portions of the present Union now claim to secede from it? All who cherish disunion sentiments are now being educated to the exact temper of doing this.

...Plainly the central idea of secession is the essence of anarchy. A majority held in restraint by constitutional checks and limitations, and always changing easily with deliberate changes of popular opinions and sentiments, is the only true sovereign of a free people. Whoever rejects it does of necessity fly to anarchy or to despotism.

And finishes with

In your hands, my dissatisfied fellow-countrymen, and not in mine, is the momentous issue of civil war. The Government will not assail you. You can have no conflict without being yourselves the aggressors. You have no oath registered in heaven to destroy the Government, while I shall have the most solemn one to "preserve, protect, and defend it."

So to sum up; you have a right to revolt if the government does not abide by it's constitution, but in current policies regarding slavery it is not doing this, so the government's duty is to enforce the union.