New Jersey had slaves before the Civil War. And the definition for a border state is “slave states that did not declare a secession from the Union and did not join the Confederacy.”). And part of West Virginia is further North than good part of New Jersey, but West Virginia was a border state. Is there any real reason that New Jersey wasn’t declared as a border state?
Good question! Although geographically speaking, New Jersey seems landlocked away from the Confederacy, there was indeed some concern in 1860 that the somewhat anti-abolitionist population of New Jersey might declare for separation (after the war, they elected George McClellan as governor, for Pete's sake). Baltimore, also geographically removed from a literal "border" relation with the Confederacy, certainly proved worrisome in this regard for Lincoln in the early days of the war, so it wasn't unreasonable to think that New Jersey might pose similar problems. The difference-maker in the whole drama was New Jersey's governor, Charles Smith Olden.
Olden had come up as a Whig, but by the late-1850s was running in the "Opposition Party," which was really just a political organization whose primary goal was to run against and beat Democrats (the party of the South). This made Olden and his Opposition Party natural allies of the newly formed Republican party (it was an enemy of my enemy is my friend kind of thing). Although Olden wasn't overtly pro-abolition, he was against succession, and a steadfast supporter of the Union.
Although he pushed for compromise early on, when all hell broke loose with South Carolina's break from the Union, Olden fell into line with the Union, and committed his state to Lincoln's side. There was indeed some concern about New Jersey's decision to back the Union early on, but when the war actually broke out, New Jersey turned into one of the most enthusiastic pro-Union enclaves of the war. Although the state had to resort to the draft later in the conflict (like many other places), its staunch support of the Union was never really in doubt. This is largely because of Olden's actions as governor, and his commitment to put his state right into the heart of the Union war effort. With the state's governor behind the Union, and little in the way of public demonstration against the Northern war effort, Lincoln never really had to worry about N.J.
New Jersey's state archive has a great section on Olden that outlines much of this history in an article you can find here: https://wayback.archive-it.org/all/20090327034055/http://www.njstatelib.org/NJ_Information/Digital_Collections/Governors_of_New_Jersey/GOLDE.pdf. Doris Kearns Goodwin also touches on the early phase of the war (late-1860 and early-1861) and these border-state questions in her book, Team of Rivals.