For example, the Spanish and Portuguese began their colonial endeavors in the 15th and 16th centuries, and the British and French began theirs in the 17th century. Yet the Dutch didn't begin colonizing what's now South Africa until the 18th century, so why did their language diverge to the point of becoming a new language when that did not happen for the aforementioned colonial powers, whose imported languages had much more time to diverge?
I am not a linguist or historian, just somewhat interested in both, but I hope that this might still be somewhat useful. I am afraid that I do not know any properly reliable sources to cite here either.
The Dutch established the colony in the Cape in the 1652 (not sure where you got 18th century from here), but lost it permanently to the United Kingdom in the Anglo-Dutch treaty of 1814. Any further expansion by Dutch-descended settlers was undertaken of their own volition (often to escape British rule), not by the Netherlands proper. This separation of the Dutch speakers in what became South Africa from their homeland (although not complete, for example the Boer Republics of the late 1800s recieved some support from Germany and the Netherlands in their fight against the British) would have added to linguistic drift. Couple that with viewing themselves as a separate people from their British rulers (which is part of what triggered the great trek inland in 1838), while no longer being linked to the Netherlands, and you can see why they would have developed linguistically differently. Despite that, Dutch was the language used in the Boer Republics, with Afrikaans being a secondary language of lower status (it is also somewhat simpler than Dutch), especially within the servant/slave communities. Afrikaans being a secondary language for use by lower status speakers goes back to the late 1700s. I have seen it referred to as originating as a "kombuistaal" or "kitchen language".
Because of the mix of people speaking it, Afrikaans has a mix of influences, with Dutch being the overwhelmingly predominant one, but also French, English, German, Malayalam Malay and indigenous African languages contributing. This leads to it having some simplifications that creoles often do. For example Dutch is gendered (like French and German), while Afrikaans uses only a single article, like English.
As regards it being considered a separate language though, that is more recent. The government of the Union of South Africa, formed in 1910, recognised Afrikaans as an official language (as a variant of Dutch) in 1925. This recognition was driven by Afrikaners, starting in the 1870s, who were people of Dutch-descent who no longer felt any ties to the Netherlands, but were viewing themselves as being their own people, born in Africa and with an African identity, and their own Afrikaans language, not Dutch.
Many creoles and pidgins have not had this recognition as an official language, so maybe it is that simple?
Edit: fixing a contributing language name.
Edit: there are numerous responses here that talk about language and dialect and sociolinguistics: https://www.reddit.com/r/AskHistorians/comments/3fmt45/why_is_afrikaans_considered_a_language_rather/
Maybe I am not understanding your question but Creole languages based on other European languages did develop in the colonial era. Haitian Creole being a good example. Is that what you are asking about? Please note that I am seeking clarification here at this point not attempting an answer to the question.