There was a very strong argument that such a law would be unconstitutional. The US Constitution doesn't give Congress the power to pass ordinary criminal laws applying to private citizens; that's the sole purview of the states (unless one of Congress's other powers, such as regulating interstate commerce, can be invoked). A summary of the legal arguments and history can be found in this law review note. (PDF) Here's a shorter summary on the House of Representatives' history website.