I'm going to try and think historically here, because this isn't really a historical question as you've posed it. There are historical components, but they push and change into the 20-year embargo period, not to mention the present day which is off limits here as a subject of analysis.

It depends on the country, quite honestly--you're talking about a vast area, even without North Africa it is over 800m people and tens of thousands of culture groups. It also isn't an either/or--Ghanaians from the Asante seat of Kumase, for example, held different ideas about where Asante identity and Ghanaian identity 'apply' in civic terms in the postcolonial era. In examples like the Hausa case, there's a third potential issue, which is identification with the precolonial caliphate of Sokoto that legitmated British claims to rule over northern Nigeria as a colony under their idealized 'indirect' rule scheme. That third set of links obviously applies less in Niger. Nigeria in particular has severe issues with religious and ethnic regionalism, which made coalescing behind a particular political party quite difficult, and helped to define both the early challenges of the country (coups and the Biafran War) as well as the federal structure and semi-devolution of certain things. However, in other countries like Tanzania (leaving out Zanzibar, which was basically coerced into union with Tanganyika and retains a separatist strain and identity), this tribalized ethnic marker is much less important because of state-building efforts rooted in the colonial era but that really reached flower with the pan-African ideas of early TANU leaders and especially Julius Nyerere. It is interesting to compare the deep concern with tribalism in Kenya with its neighbor.

So it's really all over the map, and one can't generalize, as much as educational curricula and media (including world history and related textbooks) like us to. Each example is peculiar in how ideas of ethnicity and various models of citizenship compare in strength, areas of relevance, and compatibility. I'm sorry it's not a catch-all answer, but there just isn't one beyond saying that layered identities are complicated things that sometimes cause these problems of primacy and sometimes do not.

One final note: the sense that "African borders were created arbitrarily" is a bit overblown. It is true that people of common language and culture were divided by colonial boundaries, but the entire basis for the existence of those colonies involved treaties with precolonial political entities, and European powers sometimes put remarkable effort into sussing out exactly what they were 'entitled' to claim as legitimately 'theirs.' The most arbitrary, straight borders involved areas where settled populations were very sparse (and/or colonial powers couldn't agree and so just made a line), but if you look at Hertslet's contemporary editions of The Map of Africa by Treaty you'll see this in action. Hargreaves's essay in Asiwaju's Partitioned Africans (1985) gets at that as well as some of the more arbitrary (physical feature) divisions, and Katzellenbogen's "It Didn't Happen in Berlin" in Asiwaju and Nugent, African Boundaries (1996), also complicates this narrative in interesting ways. There are surely more recent discussions of these boundary conundrums but I don't have them to hand at the moment; more focus falls on the creation of tradition, which usually doesn't question the boundary matter quite as strongly as it ought.