At some point in anglo history school for children seems not to have been universal: e.g., John Stuart Mill in the 19th century seems not to have gone to school as such (source:John Stuart Mill: A Biography) and Benjamin Franklin in the 18th century is widely cited as only having a few years of schooling (source: Autobiography of Benjamin Franklin). I also gather that there was much discussion of the pros and cons of schooling in the 18th and 19th century, that compulsory no-cost school started sometime in the mid-19th century in the US, and become universal by the early 20th century. This discussion seems to be much quieter now, with just a handful of people more or less on the fringes questioning our basic approach to schooling children (e.g., A.S. Neill, Peter Gray, John Holt).

So my question is how, why, when, and by whose ideas did we arrive at the near-universal consensus that schooling is good, and details of same (e.g. age segregation, desks, summer break, start and end ages, etc).

Also looking for any good, well-cited books or other works that summarize this history.

-EDIT: simplified question. Focus is on 18th, 19th, and early 20th century history (or earlier), and just schooling for children under age 18. Also title would more aptly read "schooling" rather than "education".

-EDIT 2: a description for people questioning our basic approach to schooling that is more objective than "more or less on the fringe" would be "small minority that inspire a strong reaction in many people ".