This mostly depends on who you are, your social status and your employment. It's also likely that this varied depending on the colonial power, i.e. France, or Germany or England. But using Kenya as an example, the colonial govt. placed extreme limitations on the freedom of movement for the majority of Africans. One reason for this was security and control. The government wanted to know where everybody was at any time. Many Kenyans had lost their land by around 1920 and so they lived in reserves. Natives needed permission from either the authorities or their employers to leave their reserves (This may have been different for people living in their own land). So they basically couldn't leave the reserves when they wanted and this restricted their movement a lot.

Further limiting freedom of movement was the Kipande system. This started with the 1915 Registration of Natives Ordinance. Basically, every native male older than 15 had to register with an administrative officer. After this they received a "kipande", identification documents that they had to wear all the time in a tin can around their necks. The system gave the administration more control over natives. Anytime a native left the farm where they were employed, the papers had to be signed. Apart from security, the restricted movement allowed the administration to utilize the natives as a very cheap labor source through a forced labor system. The native reserves and the kipande worked together to enforce this. If a native was not at work, then he was only expected to be in the reserve. before the Kipande system was introduced, there was a lot of rural-urban migration by educated Africans as they sought salaried work, so it is possible that some of them may have gone to other colonies. But the Kipande system put a stop to that and Africans in Kenya would have been unable to leave the country or even travel between towns.

That being said, the systems loosened as the 1950s approached. Some Africans soon had their own farms and businesses and there was some freedom of movement (maybe even to other colonies, not really sure on this). However, the 50s were bad times for Kenya. Because of the Mau Mau movement, the Governor (Sir Everlyn Baring) declared a state of emergency and this led to the detention of hundreds of thousands of people from certain tribes (mostly the kikuyu), extremely limiting their movement. So again, Many Africans could not travel or leave the country.

To put it briefly, movement for Africans in colonial Kenya depended on many factors but it was extremely limited for the most part.

A bunch of sources:

On native reserves, forced labour and the kipande system: [Bruce Berman 1990] (https://books.google.co.ke/books?id=kVE4xokuFSkC&pg=PA147&dq=Kipande+system+in+Kenya&hl=en&sa=X&redir_esc=y#v=onepage&q=Kipande%20system%20in%20Kenya&f=false)

On the colonial economy and the forced labor system: [Swainson 1980] (https://books.google.co.ke/books?id=-pzxavse89gC&pg=PA33&dq=Kipande+system+in+Kenya&hl=en&sa=X&redir_esc=y#v=onepage&q=Kipande%20system%20in%20Kenya&f=false)

A primary source mention on forced labour and native reserves: 1928 national assembly record

And a good explanation of native reserves and the colonial land system

Again, this is for colonial Kenya. The system might have been similar for Uganda and Tanzania but it was certainly different for other parts of Africa and other colonial powers.