It's a shortening of the term "private soldier". My understanding is that it denotes a soldier that is not an officer (neither commissioned nor noncommissioned). In other words, everybody below corporal (the traditional rank, not the current usage which is more like an experienced/senior private).

This article seems to have a fairly accurate explanation of many military ranks: http://www.wordorigins.org/index.php/more/896/