No, it was not illegal. As stated in the U.S. Constitution,

Each state shall appoint, in such manner as the Legislature thereof may direct, a number of electors, equal to the whole number of Senators and Representatives to which the State may be entitled in the Congress: but no Senator or Representative, or person holding an office of trust or profit under the United States, shall be appointed an elector.

This gives very broad latitude for states to create rules for selection of electors for President. Of the U.S. states in 1860, only South Carolina had no form of popular vote for President (rather, the state legislature in that state selected the electors). Lincoln recieved .93% of the vote in Kentucky in 1860, 2.48% of the vote in Maryland, and 1.13% of the vote in Virginia, suggesting weak Republican organization there. Republican organization in states further South was no doubt even weaker. Prior to 1888, states didn't have ballot access laws; rather, ballots were printed by political parties. The first government printed ballot law was enacted in Massachusetts in 1888; almost all states in the country followed suit over the next twenty years.

The last major-party presidential candidate not included on the ballot in a U.S. state was William Howard Taft in South Dakota and California in 1912.