Look into the passing of the Homestead Act of 1862. There were several previous acts that allowed US citizens to lay claim to Government land by paying nominal fees. But the Homestead Act is generally seen as the most important government initiative that encouraged westward expansion.

Basically, though: Western lands were in a kind of limbo because of the issue of slavery (will Western states be slave or free?), so the Homestead Act of 1862 is wrapped up in the Civil War as well. Lincoln allowed for any US citizen who had never taken up arms against the US government (read: not a Confederate) to claim 160 acres of government land as long as they would improve upon it (i.e. farm it). That's how many huge tracts of Western lands ended up divided among numerous private citizens.

Finally - a note on semantics -- private land wasn't allocated; public land was. Private land that was held by individuals would have been bought and sold the same way that it is today. Also, there really wasn't any "unclaimed" land. It would have been variously held by private individuals, government entities (US or non), or Indian tribes. Also, the idea of the "first Americans" entering the west is itself somewhat spurious. What we now consider the Western states were held by Native Americans, Spanish, Mexicans, and French among others (remember, Alaska, aka Seward's Folly, was bought from the Russians). New Mexico and California in particular were areas with longstanding Euro-Mexican settlements. There were trade networks that went back hundreds of years. I only mention all this because I think there's this idea of a vast, untamed wilderness that some Anglo-Americans walked out onto one day to lay claim to, and that's not really what it was like.

Anyhow, that's my initial two cents. I'm sure some other historians will have more to add. I'm not exactly an expert on the subject, but I've spent the last three years of my career in the West, so I have come to understand a bit more about it.