If you have to be born a US citizen to become President, how did any of the men before Van Buren even take office? I guess that there was some sort of exception but what were the technicalities about it?
From Article II, Section 1 of the Constitution:
No Person except a natural born Citizen, or a Citizen of the United States, at the time of the Adoption of this Constitution, shall be eligible to the Office of President; neither shall any person be eligible to that Office who shall not have attained to the Age of thirty five Years, and been fourteen Years a Resident within the United States.
The Founders wanted to limit presidential office to "natural born" citizens (whatever that means -- it's never been tested in court) to avoid foreign influence in elections, or a foreign person becoming President. The 14th Amendment starts thusly:
All persons born or naturalized in the United States, and subject to the jurisdiction thereof, are citizens of the United States and of the State wherein they reside.
That text has generally been held to confer citizenship (both state and national) on anyone born in the United States, including its territories; also, people born outside the U.S. to citizen parents are considered citizens.
At the time the Constitution was written, this would also, of course, have excluded enslaved persons and Indigenous people from the office, but the Founders were less concerned about those people than they were of Europeans attempting to take control of the country.
What exactly is meant by a "natural born citizen" in this context is a fairly open question; there have been attempts to bring cases before the Supreme Court regarding this question, usually regarding candidates that have one parent born outside of the United States, but they have been dismissed for lack of standing. The 20-year rule unfortunately prohibits most discussion on that.