If you look at this map of the Roman Empire's western provinces, you'll see that the area which is currently the Netherlands and Belgium is called Belgica. Since Belgica is the Latin name for the Netherlands themselves, the New Netherlands in Latin becomes literally the new Belgica.

As for giving it a Latin name to begin with, Philip Ayres in Classical Culture and the Idea of Rome in Eighteenth-century England talks about seventeenth and eighteenth century neo-classicism in England, and similar trends in thought were happening throughout Europe with the growth of imperialism, as countries like England and the Netherlands looked to the past for a justification and a model for their expansion.

As a Belgian it confuses me why they called the New Netherlands in latin Nova Belgicum.

Why would it? Don't forget that at the time there was no 'Belgium', except for the occasional use as a Latin name for the Low Countries/United Provinces. The idea of a 'Belgium' separate from 'the Netherlands' (in its widest sense) didn't exist back then.