People who lived in caves and rockshelters kept dry pretty much the way you would in a home you built from scratch. First, and most importantly, they chose their caves deliberately, picking shelters that would avoid direct wind, minimize flooding or seepage, and maximize heat or cooling (depending on the season).

Second, they would put down "flooring" to help minimize dust and soak up moisture. A common choice for "flooring" would be woven matting or loose, dried straw, sand and ash, or clay. Matting, straw or sand/ash could be removed and replaced. Clay surfaces could be "refloored" by adding a new layer of clay after a surface got too damaged or dirty. Archaeological excavations often find layer upon layer of deliberately-designed floor materials in prehistoric caves.

Third, people would often modify the natural features of their cave to make it more livable. They would "build up" the floors of caves with clay or stone to create separate living spaces and "platforms" which would be higher than the natural cave floor, keeping their sleeping and storage spaces separate from messy workspaces where gunk from fire or food processing would accumulate. They also would improve the natural cave opening by creating channels to funnel away water, and building up stone walls to extend or block up part of the cave's mouth, or creating windbreaks with hide or fabric stretched across wooden posts.

I'm in a hurry right now but I'll add some specific examples later.