I understand that ancient civilizations knew generally about the passage of time through seasons and by the rising and setting of the sun and moon, but when did the first civilization realize that a year was 365ish days and that a day could be broken down evenly into hours, minutes, and seconds? And when did the world come together to say "once every four years -- except on years that end in 00 -- we will have an extra day to make up for the fraction of a day we leave out every other year"?
The knowledge that the tropical year was 365ish days may have been known as early as 8,000 BC. The Warren Field monument aligns with the winter solstice as well as the lunar cycle. By around 3000 BC there are more 'calendar monuments' found (such as Stonehenge) that align with the solstices, and any culture sophisticated enough to build them could certainly count days between solstices.
The concept of inserting days into the calendar goes well back before the '1 every 4 years'. They came about because the lunar 29 day cycle doesn't fit evenly into the 365.25 day tropical year. So a number of cultures developed means to, with some success, work around the problem. The Iranian and Egyptian calendars used 365 days, and the Egyptians were probably the first to try to insert a leap day every 4 years. Since our current calendar system traces its roots back to the Roman system, I'll discuss that.
The Roman calendar had, for quite some time, 355 days. The additional days were made up by inserting an intercalary month between February and March. Ideally this would create alternating years of 355 and 377 or 378 days. However, the Pontifex Maximus (high priest) was in charge of determining when to insert the intercalary month. At various times the extra days would be inserted to extend a political ally's term of office, or omitted to shorten a political opponent's term. Other times, particularly when Julius Caesar (who was Pontifex Maximus) was fighting the Civil War, the intercalary month was forgotten.
By the time Caesar consolidated his power, the Roman calendar was almost three months off. So winter festivals occurred in the fall, and summer festivals occurred in the spring. To correct this, Caesar not only added the normal intercalary month, but also added an additional 67 days, making the year 46 BC a whopping 445 days long. In addition, Caesar reformed the lengths of the months to the pattern we're familiar with now (30 days hath September, etc). He also instituted the leap year, adding a single day every four years. This 'Julian Calendar' almost brought the calendar in line with the tropical year.
However, the priests didn't implement the reforms correctly, adding a day every three years. Augustus was forced, 12 years later, to correct the calendar once again. A minor change was implemented under Pope Gregory XIII in AD 1582. By this time, the date of Easter had migrated 10 days since the Council of Nicea set it in AD 325, since the "1 day in 4 years" is just a bit off. Aside from cutting 10 days out of the year 1582, they also adopted the practice of having years divisible by 100 NOT be leap years, unless also divisible by 400 (So 2000 was a leap year, while 1900 was not). This Gregorian calendar was adopted in Catholic areas in the 1580's, while Protestant states generally began adopting it in the 1700's, with Britain and her colonies adopting it in 1750. Eastern Europe held on to the Julian calendar until the early 20th Century (and most Orthodox Churches still use Julian to date religious holidays, so Orthodox Christmas occurs on 7 January by the Gregorian date).