You would have to carry your freedom papers or be able to produce them. You could also have local people testify that they knew you and always knew you to be free. That's basically how it worked. Frederick Douglass traveled North using papers he bought from a kind sailor. Other people might forge papers to gain freedom. But this was basically how it happened....

Yes, they would need to carry their free papers with them all the time. There were people who were eager to catch escaped slaves, both for the entertainment value and the financial reward for catching an escaped slave. Such people would demand to see the papers of any black person they saw unattended. Slaves needed to have a paper from their owner giving them permission to be where they were.

In practice, it was also not too difficult to enslave a free black person at this time by claiming the person was an escaped slave. Look at Solomon Northup, the man who wrote 12 Years a Slave. He was beaten severely for claiming to be free. But beyond that instance, laws were passed that were heavily favorable to slave owners and put the burden of proof that you were free, onto the accused. This was a major point of contention between slave and free states. In free states, black children in particular were seized and taken to slave states and passed off as slaves who had been born in the slave state. What defense could children provide? But adults, too, were taken, as Solomon Northup's abduction demonstrates. His was not an anomaly, but was quite frequent. Due to the Fugitive Slave Acts, a slave catcher could just grab any black person they saw, and all the slave catcher would need to do would be to show a deed proving that a slave meeting that description had escaped and was being sought. The accused slave was not allowed to testify on his own behalf, it being a property case--it would be like cattle being expected to testify regarding who rustled them. Other members of the court might be in collusion and delay, delay, delay in cases to prevent any action by abolitionists until after the purported slave was safely on the way back south. And the court hearings where this took place were usually kept pretty quiet so the slave had no ability to get someone to testify that they knew him and he was free and a resident of that area of long-standing. There are many cases of abolitionists getting wind of such a hearing and storming into the courthouse, grabbing the accused slave, and carrying him away to safety. There were certain judges who were known for just rubber-stamping any case where a slave-catcher claimed he'd found an escaped slave. There were abolitionists who worked tirelessly to prevent people, whether born free, manumitted, or an escaped slave, from being taken to slavery. But the speed with which these court hearings were conducted were quite deliberate in getting the accused onto a ship to a slave state before the abolitionists and friends had even heard about the person being grabbed. You know how people have claimed that the cause of the Civil War was "States Rights"? Well, the rights they were talking about were the rights to their slaves--northern states were very oppositional in a lot of cases to returning escaped slaves, and passed a lot of laws declaring things like if a slave had legally (not escaped) been living in New York for 6 months, then he was free. Or if a slave were brought to a free state, he was immediately free. Slavery's defenders pointed to the Constitution as granting them the right to their property, regardless of what state they carried that property into. The supreme court agreed with them. The Fugitive Slave Act of 1850 was particularly harsh, making it a matter of federal law to retrieve escaped slaves, and no number of years of living in freedom would keep you safe. If you were a woman who had lived as a free woman for decades in a free state, but were accused of being an escaped slave, then you could immediately be carried back to the south, and all of your children with you, because even though those children were born in a free state, they were legally slaves because their mother was a slave.