This is actually something that's spelled out in Article I of the Constitution:

Clause 2: Immediately after they shall be assembled in Consequence of the first Election, they shall be divided as equally as may be into three Classes. The Seats of the Senators of the first Class shall be vacated at the Expiration of the second Year, of the second Class at the Expiration of the fourth Year, and of the third Class at the Expiration of the sixth Year, so that one third may be chosen every second Year; and if Vacancies happen by Resignation, or otherwise, during the Recess of the Legislature of any State, the Executive thereof may make temporary Appointments until the next Meeting of the Legislature, which shall then fill such Vacancies.

When the first Senate was assembled, the assembled senators essentially drew lots for their classes, with the caveat that no state would have two senators from the same class (so that you never have two Senate elections in a year in one state, but they alternate two- and four-year elections). When new states are added, which last happened in 1959, the senators are added to classes with an eye to keeping them as close to the same size as possible and each new senator is placed into either of the classes by a coin toss.

Apologies for not being able to better source this, but I've been reading Caro's Lyndon Johnson biographies and I think this comes up in Means to Ascent, but don't quote me on it. It's something I remember back from working in the Congress a couple decades ago.