Well, the notion of a regular duple (and its multiples) arrangement of musical phrases was certainly not an invention of modern pop music. Edward T. Cone pointed out way back in 1964 that "in Romantic Music [...] one can find long stretches in which the measures combine into phrases that are themselves metrically conceived [that is, in which each measure is like one beat in a larger 4-measure "hypermeter"]. It is here [...] that we can justly speak of the tyranny of the four-bar phrase." People have likewise long claimed that 18th century music is based on "symmetrical" phrases, though in practice, a great deal of the supposed symmetry is stretched and distorted beyond recognition by various techniques of stretching or compressing phrases. But I do think that the four-bar phrase is conceptually basic to the eighteenth century as well. In the same way that many great, eloquent, and complex sentences can be reduced to a simple background of "noun + verb," a great many phrases in classical and jazz music are very artful manipulations and elaborations of a relatively simple 4-bar background structure. So this is music that plays off of a basic four-bar unit, but rarely articulates it in a very straightforward way. Perhaps this is what you meant by the way classical and jazz music sound very rhythmically different from pop music. (If you want, I have some favorite examples I can elaborate for you on this point if you want to see this kind of thing in action, but I don't want to overcrowd this post since it's a little tangential).

Even well before the eighteenth century, regular, duply-based musical groups appear frequently in western music. Broadly, there are two sources for this.

1.) In many courtly dances (such as, say, the Pavane and Galliarde), duple groups are quite omnipresent, in part so as to easily coordinate with dance steps. That is, having predictable phrase endings helps to coordinate complex group choreographies amongst the dancers, helping them know when to begin, end, and change moves

2.) Vocal music written for metrically regular quatrains of poetry. Whenever there is a style that wants to match that poetry to a rhythmically straightforward and syllabic music, such quatrains will tend to generate duple music. (This is especially true for e.g. Italian song, since there are some special demands of metrical placement that Italian prosody demands).

So, in fact, there may never really have been an "advent of the 4-bar concept." We've been living with it for a very, very long time, and it comes in at multiple angles.

But of course, that does not mean that there have never been alternatives. There are any number of ways that alternative rhythmic structures may be generated. Starting from text, again, if you are working with metrically irregular text or poetry that isn't organized into four-verse stanzas, then that may well prompt non-duple music. Much gregorian chant is like this. Thus, check out this Latin version of a bit from Psalm 93, which if you count the syllables in, you will see quite clearly that it is not a metrically regular poem.

Viderunt omnes fines terræ
salutare Dei nostri.
Jubilate Deo, omnis terra.
Notum fecit Dominus salutare suum;
ante conspectum gentium
revelavit justitiam suam.

So even the most straightforward music for such a text would essentially have to be irregular. And then there's the fact that you can extend the amount of time you spend on different syllables by, say, drawing them out over many notes (a melisma). This is exactly what the gregorian chant for the above text does: in it, you can hear the first syllable of "omnes" and "terræ" in the first line being drawn out extensively, resulting in a not-very-metrical phrase.

Coming at it from the instrumental domain, A lot of early instrumental music that was not based on dances was based instead on taking a little one-measure thematic idea and "spinning it out," repeating it and transposing it as you added more musical stuff around it to make harmonies. This was a more "generative" approach: rather than starting from a broad metrical framework and slotting music into it, you started with a small idea and built it up to see what you could get out of it. And as a result, you could often generate a whole bunch of strange, irregular phrases with this method.

There are many other ways of doing things too. People in like the 13th century experimented with all sorts of crazy ways of structuring musical time. Because they essentially had just figured out the notational means of clarifying how multiple parts playing at the same time are supposed to line up (in other words, they had just figured out how rhythmic notation worked). And so they went pretty nuts thinking of ways of manipulating that newfound technology. If you really want to go down a rabbit hole, check out the "Ars subtilior," which produced some of the most bonkers, esoteric music I've ever heard!