What I mean is, when the human population was very low, did they all speak the same (or similar) language when they were located around the same area before humans started venturing out of Africa?
While we can't prove that a "proto-human language" existed, there is enough evidence supporting this hypothesis-- such as hyperconserved words-- to drive ongoing research into a possible first human language. Researchers have use (and continue to use) a variety of methods to try to identify 1) when humans spoke this first language, and 2) what that language might have been like. For a well-written and accessible overview of the subject, I'd recommend the book The First Word by Christine Kenneally.
The estimates for when humans all spoke a shared language vary: studies of phenome diversity over time give an estimate between 150,000-350,000 years ago, which is largely consistent with estimates that have existed since the theory of language monogenesis appeared in Western academia in 1905. For context, this places a universal human language around the same time as the first archaeological evidence of Homo sapiens as a distinct species. However, some researches-- such as anthropologist Dr. Richard Klein-- believe this may have been as recent as 50,000 years ago. This would be right around the emergence of "behaviorally modern" humans.
A 2011 study posited that this language would have had a Subject-Object-Verb (SOV) word order (this is still the most prevalent word order, with 45% of the world's languages being structured this way). SOV languages also show other similarities in word order. Studies of phenome diversity modern African languages (see above) support the idea that this proto--human language would have had a complex and varied set of consonants, likely including clicks.
Several linguists (most famously Noam Chomsky and Joseph Green) have attempted to come up with a list of universal characteristics for all human languages; however, the idea that "linguistic universals" even exist is still controversial.