I'd like to answer this question specifically from the perspective of Ancient Greek. I think that the more general answer I give is likely to be accepted by experts in other areas, but I'm open to being proven wrong there. Some aspects of this question are probably better suited to our friends at /r/asklinguistics, but I'd defend that historians always have a stake as well.

To get straight to it, my goal with this post is to stress that we can't make the distinction you suggest at all. That is, there is no appropriate measure, even given evidence that we do not possess, by which we could establish a specific moment for the transition between modes of the Ancient Greek language used in the Classical Period and the thing we call "Koine."

I think it would be useful to make a few points about the array of dialects represented within the overarching language first. To make this point best, it will be handy to take a quick detour to the Renaissance to examine the impact of the printing press on the phenomenon of dialect. Imagine that the default setting for any pre-industrial country is for the population to divide into quite small dialect pockets operating on the scale of micro-regions and valleys, villages and market towns. Without a shared and widely disseminated literature, there occurred a natural drift in the daily language of peoples who, though united by history and the broad similarities of their speech, picked up new words and added textures to their vowels along the way. With the printing press, however, we see a gradual homogenization of major European languages as a single dialect. The classic example of this is the transformation of the Saxon dialect of German into the principal and dominant form of the language following its use in Martin Luther's translation of the bible.

Ancient Greece, of course, existed a bit before Martin Luther. I want to pull out a couple of examples from the epigraphic record to demonstrate just how much versions of Greek could differ over even very short distances. I'm going to start with a slightly extreme example, since that's just fun. My example will be Eretria, one of the major city-states occupying the island of Euboea to the north of Attica. Now, the Eretrians are known for a few things when it comes to their way of speaking Greek. Probably the most famous is a phenomenon called "rhotacism," which refers to the tendency of a speaker or dialect to transform other sounds into "r." The Eretrian dialect displays intervocalic rhotacism, meaning that the letter rho, the "r" sound, often appears in place of other letters, especially sigma, in the middle of words. To give an example, one of the Ancient Greek words for "food" is sitos (σῖτος). A related word to this, sitesis, meaning "the right to be fed [at the expense of the city]," appears in the form that follows in an inscription of roughly 340 BCE: siterin (σίτηριν) instead of the form that we would find at Athens across the Euboean Gulf (sitesin, σίτησιν). Another example, which Eretria shares with several other Greek cities of the Ionic branch, is the tendency to pronounce two taus in situations where two sigmas would appear. To take a fun example, the name "Narcissus" would be written Narcissos (Νάρκισσος) in a "standard" (more on this soon) Koine Greek of a later period. At Eretria in the 4th Century BCE, however, it was written and doubtless pronounced as "Narcittos" (Νάρκιττος). Both of these features disappear for the most part at Eretria toward the end of the Fourth Century BCE.

That of course seems to tie in well with common narratives linking the spread of the Koine version of the language with the conquests of Alexander the Great and the accompanying political domination of mainland Greece by various Macedonian warlords through the start of the 2nd Century BCE. However, it's a bit more complicated than that. A good example of how a city-state could lag behind the overall trend towards the adopting of a "koine" variant of the Ancient Greek language is Rhodes, which spoke a version of Greek belonging to the Doric group that typified the speech of many peoples in the Peloponnese and elsewhere, like the Spartans. The thing is, the Rhodians were still using their homespun variety of Doric Greek in public documents (that is, inscriptions) through roughly 200 CE. That's right, CE. Think back now to the digression that I made about the printing press. The cultural and intellectual assimilations that occurred in the wake of the conquests of Alexander in some respects had a similar effect in homogenizing the Ancient Greek language; however, it was ultimately, much less powerful of a drive linguistically than the centrifugal effect of localism and the relatively low contact between regions in the ancient world compared to the ties of the early modern period. Groups of Greeks like the Rhodians kept a version of their dialect notably different from a pure version of Koine, if such a thing ever existed, on through the high days of the Roman Empire. As for the moment when Koine Greek became more or less prominent, even granting this kind of exception, I'd say a few other things.

When people who study Ancient Greek documents, primarily epigraphists, say to themselves "that's Koine," there are a variety of features of the language that they use to support this instinct. To produce a few examples for you, it's very common for decrees inscribed on stone from the Ancient Greek world to begin in part with a clause along these lines: "So and So introduced the proposal [that will follow in the rest of the inscribed document]." In Ancient Greek, the word for introducing the proposal would, in Classical Greek, be eipon (εἶπον). However, something interesting happens as you get into the Hellenistic Period, at different times at different places. This requires getting a bit into the weeds. So, there are two different ways to form the type of past tense verb (the aorist) that "eipon" belongs to in Ancient Greek. The method of word formation that "eipon" belongs to is the less common of the two. This means in practice that "eipon" is using a different ending (= -on) attached to its stem (= eip-) than would be typical for other types of similar verb that Ancient Greek speakers used. In the Hellenistic Period, like I started to say a second ago, a change occurs. Ancient Greek speakers seem to have started applying the endings of the more common past verb type to the less common type that eipon had previously belonged to. Here is what that looks like. It is a shift from the previous form eipon (εἶπον) to "eipan" (εἶπαν). That's it. Just one letter. Another example, also involving a single letter, would be the tendency of the verb meaning "become" to shift from the spelling gignomai (γίγνομαι) to ginomai (γίνομαι).

To conclude, Ancient Greek was always, at all periods, comprised of divergent and competing dialects. While a stage of the Ancient Greek language that we call Koine did begin to appear at the end of the Fourth Century BCE, it did so in stages, adding its features gradually and at a dissimilar rate in dissimilar forms at different places at different times. For this reason, it must be regarded as a process rather than something that can be spoken of as commencing at a defined point in a complete and absolute form.

Some Bibliography

Bean, G. E. and P. M. Fraser. The Rhodian Peraea and the Islands (1954).

Buck, C. D. The Greek Dialects (1955).

Knoepfler, Denis. “Loi d’Érétrie contre la tyrannie et l’oligarchie,” Bulletin de correspondance hellénique (2001): 195-238. (My example of rhotacism is from the editio princeps found here.)

Knoepfler, Denis. La Patrie de Narcisse (2010)