In what language? In what meter? The earliest written verse we have isn't rhyming and I'm not sure if Sumerian verse even has a consistent meter (you'd have to ask someone who studies the Near East). Because of case structure it's extremely easy to rhyme in Greek and Latin and rhyming lines were considered poor form, they never had a set verse form in which rhyme was important. The answer to the question depends entirely on what language you're asking about

As a fun fact, but not to be construed as the definitive answer, old English poetry was exclusively alliterative instead of rhyming, since it helped the early bards to remember the works on a verse by verse basis. Cædmon's Hymn composed in the mid 7th century is a fine example of it, and you can hear the verse by verse alliteration in it here. Beowulf is also a fine example of it of course, but it comes later than Cædmon's Hymn.