I'm not interested in their colonial history. Looking for anything relatively accessible but particularly on how the countries were formed.
I presume in English?
History of the Low Countries. E. Lamberts
The Low Countries: A History. Anthony Bailey (not a huge fan of this book but there are precious few books about this subject in English)
For Belgium specifically:
Belgium: Long United, Long Divided by Samuel Humes
How can one not be interested in Belgian History by Marc Reynebeau, Geert van Istendael and others.
If you really want to learn about Belgian politics and the complexity of the Belgian political system you can pick up: Political History of Belgium by Els Witte but fair warning: this is the book used in the university course and while it does a great job of laying out the whole history it is very factual and very dry.