First, let me note that this is really a huge question. Africa is the world's second largest continent and it's incredibly diverse. And the Victorian period was really long, from 1838 to 1903. I'll try to give a partial answer focusing only on tourism and on the later Victorian period (after about 1885 or so) and on sub-Saharan Africa. North Africa and Egypt are their own particular thing.

So, there wasn't much tourism in Africa at all during this period. And what little there was, happened in just a few places. When we think of colonial white tourists in Africa, we're probably thinking of the period from around 1900 to 1940. Most of the tropes that are associated with "old time" African tourism -- "great white hunters" on safari, caravans of native bearers, a luxury hotel at Victoria Falls, biplanes -- are early 20th century, not 19th century. Before 1900 there weren't a lot of whites in Africa for any reason, and most of those were merchants, missionaries, or engaged in some colonial enterprise like mining, plantation farming, or rubber extraction.

Keep in mind that before 1900, mass tourism barely existed, and tourism to someplace as distant as Africa was almost strictly limited to the wealthy plus a few eccentrics. Also, most of sub-Saharan Africa was really dangerous to visit well into the 1880s! That's because Europeans didn't really get a handle on tropical diseases until around then. As late as the 1870s, you had large numbers of Europeans dropping dead from various pestilences. A while back I visited the island of Goree, just off the coast of Senegal. There's a memorial there to thousands of people -- including hundreds of Europeans -- who died in the yellow fever epidemic of 1878. Would you want to visit the modern Congo in the middle of an Ebola epidemic? Or Wuhan, China right now? That's how most Europeans felt about Africa before 1880. People did go there -- but they saw it as dangerous, so they didn't go without a good reason.

Also, large chunks of Africa were still no-go areas because of the violence associated with colonialism. You really wouldn't have wanted to be a tourist in South Africa during the Zulu Wars. And you really REALLY wouldn't have wanted to be a tourist in the Congo during King Leopold's genocidal rule. Lots of parts of Africa were pretty peaceful during the period 1880-1900, but lots of other parts weren't.

So, up until around 1880 tourism in Africa was sort of like tourism in Antartica today: a few people did it, but they were mostly wealthy and mostly stuck to the coasts, plus there were a few eccentrics doing who-knows-what.

After 1880 you do see a bit of tourism, mostly built around safari hunting. "How and why did safari hunting become popular" is its own interesting question, but let's just note that it did. But while it certainly existed in the 1890s, it didn't really take off until after 1900.

Another thing to keep in mind is that up to 1900, most of Africa was still really hard to get to. The automobile was unknown. Railroad construction was just getting under way. Most of the continent required a long walk inland from a coastal port, and there weren't that many good coastal ports. Telegraph lines existed, but only to selected locations, so it was easy to simply disappear.

Okay so, to answer a few specific questions:

Were natives used as servants? -- Of course. That was true pretty much everywhere under colonialism, but it was especially true in Africa. Why import a European, at vast expense, when you can train an African to do the job? You can pay him almost nothing! All the servants were Africans. And despite constant grumbling by the colonials about how lazy and unreliable the Africans were, they found no difficulty in running all sorts of enterprises, from railroads to luxury hotels, staffed almost entirely by African workers.

Was it seen as dangerous and scary, or interesting and adventurous? -- Mostly dangerous and scary (though sometimes wild and interesting) up until around 1880, then gradually less scary and more exotic / cool.

Did anyone ever vacation to Africa for pleasure rather than business? -- A few people, and mostly late in the period, and mostly to a short list of relatively safe, peaceful destinations, particularly South Africa, Senegal, and Kenya. Since Africa is huge, there are some interesting exceptions. For instance, the avant-garde French poet Arthur Rimbaud spent the 1880s bouncing around East Africa, doing everything from gun-running to amateur exploration to becoming one of the first Europeans to export real Ethiopian coffee from Ethiopia. And there were a few travel writers and amateur naturalists; those guys pop up everywhere. In the 1890s, a seemingly ordinary British woman named Mary Kingsley simply decided to go to West Africa. She ended up traveling thousands of miles deep in the bush and writing two books, one of which -- _Travels in West Africa_ -- was an instant best seller and is still worth reading today.

How common were towns built for European visitors/residents? -- Actual towns? Quite rare. There are a few towns built by Europeans to live in, like St. Louis in Senegal. But even there, the locals outnumbered the Europeans. There were regions of European settlement, like the Boer lands in what's now South Africa, and the highlands of Kenya and Tanzania which were claimed by English and German settlers. But, again, locals outnumbered Europeans even there. Think about it. Why would a European move to live in Africa? Well, because you would have cheap abundant labor to do the farm work, dig the mines, lay the rails, shovel the coal, pave the roads, and do all your housework. That was the whole point of being a European colonist!

Later in the 20th century you would get colonial segregation, most famously in the form of South African _apartheid_. But that was a 20th century thing. In the 19th century, most Europeans in Africa lived surrounded by Africans, and there weren't really any all-European towns or neighborhoods yet.

I'm not sure if this answer is what you're looking for, but I hope it helps.