Languages spoken in ancient times are not inherently any different than languages spoken today, so yes, ancient languages did have concrete rules and syntax in the same manner as modern languages. This is essentially the principle of uniformitarianism, foundational to virtually all fields of science, as applied to linguistics. Uniformitarianism holds that the laws and processes of nature are invariant throughout time and space, and that the same basic methodology can be used to study and describe natural phenomena across time and space. Now, of course, language itself has not existed for all of time; language as we know it is linked to the biological, intellectual, and cultural capacities of human beings. But human beings who lived 10,000 years ago were the same biological species that we are now, so we do not expect - and we do not find - that language operated in any fundamentally different way in previous eras of human society.

And you are correct in thinking that most people are not usually aware of these rules unless taught. However, your post seems to be based on the assumption that one must be aware of the rules of language in order to know and use them. This is actually a misconception about the nature of language.

Every healthy human being with functioning legs knows how to walk. However, virtually no human could tell you specifically how one walks beyond "You put one leg in front of another." What muscle groups do you contract and release, and in what order? What angle do you position your legs, feet, and toes in? We do not need to be aware of this consciously, because we know it unconsciously, encoded into our brains at a very deep level.

Using language is similar. How do you produce a question in the English language? What methods are available, and what are the specific steps you take to produce a sentence using each of them? What happens if you want to add negation to a question? Most speakers of English could not at all explain this, and we don't need to, since it is has been encoded in our minds at a level not normally accessible to conscious awareness since childhood.

Thus, we don't need to "agree" on the rules of language with other speakers of the same language, because we are not consciously aware of most of them - especially the more complicated ones - to begin with. Also as a consequence, we don't need to be taught how to use language in school, since it is something we pick up naturally in childhood. What most schools in the Western world teach in English, language arts, or grammar classes has very little to do with the rules of language. Most of it has to do with teaching children the rules of writing and basic genres of writing, such as letters, essays, etc. Sometimes schools even teach children falsehoods that are presented as rules, such as "You can't begin a sentence with a conjunction", even though these "rules" are not observed even in formal literature, and never have been. Or, sometimes, schools teach children how to use the standard language of a country, instead of their local dialect, in the manner one would teach a foreign language (although most schools still present this as a matter of "correct" or "incorrect").

People in ancient times had similar practices, where some people would tell others that such-and-such way of using language was right or wrong. Often this happened in schools, but like today, it very often had nothing to do with the actual rules of language that speakers inherently knew from growing up in a human society that used language. I'm not an expert on the grammatical and educational traditions of any ancient society, so I can't go into more detail there.

So, yes, ancient languages did have concrete rules; no, as with today, most people would not have been aware of them; however, there was and is no need for people to consciously "agree" on rules of language, as all of the speakers of a given variety of language would have deeply and unconsciously internalized a broadly similar (though not identical) set of linguistic rules from an early age.

As for a source, these are principles so foundational to the field of linguistics that I am not sure how to cite them. I hope the mods can permit me just to say that these are topics so basic that they would be covered on the first day of any Introduction to Linguistics class.