It depends a lot on what you mean. If you pull up any map of the Roman Empire, you'll see pretty much instantly that their power extended over the entire northern coast of Africa, from Egypt to the north of modern Morocco. After all, Rome's first major territorial gains outside of Italy came after the Second and Third Punic Wars against Carthage, which controlled the area of modern Tunisia and parts of Algeria and Libya. Subsequent conquests included the kingdoms of Numidia and Mauretania, which ruled most of coastal Algeria and Morocco. Finally, there's eastern Libya, which split off from the main Ptolemaic kingdom in Egypt and was bequeathed to the Roman Republic in 96 BCE.

So, on a very technical level, yes, they went beyond Egypt, but it's probably safe to assume you could see that for your self. The real question that you, and others, have wondered, is why didn't they expand further into Africa beyond that strip of northern coastline. If you look at the region, the reason is obvious in most directions. From the Nile River all the way to the western coast on the Atlantic they faced the nearly impenetrable barrier of the Sahara Desert. Sparsely populated, treacherous to cross, and lacking valuable resources, the Romans had no need, desire, or even real ability to try and cross the vast expanse of sand and heat.

To a casual observer just looking at a political map, it looks like they should have been able expand as far down the west African coast as they wanted, but there is a reason very few European ships made that journey prior to the 15th century CE. The North Atlantic Gyre, specifically the Canary Current and North Equatorial drift, made navigation almost impossible. The current pushes south along the west African coast to around Senegal and then starts pushing ships out into the Atlantic unless they hug the coast, but if they do that, the current is still pushing them south. So the journey to the Ivory Coast was practically a one way trip. They could send ships there, but they could not make it back.

So that just leaves the eastern part of Africa. We can't really say with certainty why the Romans never tried to expand their empire southward along the Nile. One very plausible reason is that it just started to get too far away from the centers of Roman power around the Mediterranean, and they were already stretching their resources trying to control or expand their European borders. Unlike the Rhine and Danube borders with Germania, the southern boundary of Egypt was not heavily contested. The Romans regularly had to deal with Germanic tribes crossing the northern boundaries during the Migration Period, but the southern Egyptian border with the Nubian kingdoms was very well established, and rarely contested.

There were occasional border skirmishes or raids into Roman territory and counter raids to the south, but never much serious. Strabo describes one invasion of Roman territory by the Kushite kingdom of Meroe in the 1st century BCE that was repeatedly repulsed, but never retaliated against. The conflict actually generated a peace treaty that established favorable trading conditions between the Kushites and the Romans. In later centuries, Rome seems to have used Meroe as a client kingdom/buffer between Egypt and more hostile Nubian groups further south, basically passing the buck for defense of that southern border in exchange for some financial or political support.

Then in the 4th century CE, most of Nubia, including Meroe was conquered, but not by Rome. They were conquered by King Ezana of Axum. The Axumite Kingdom was an empire that originated in modern Ethiopia. So now another powerful kingdom was on the southern Egyptian border right around the time that Roman power was really starting to wane. We know that there was diplomatic and mercantile contact between the two, and even a large enough Roman expat population in Axum for King Ezana to carve some of his monuments with a Greek inscription in addition to the local Ge'etz and Arabic languages. The Axumite kingdom ultimately outlasted Roman rule in Egypt by more than a century, though it had receded away from the Egyptian border by the time of the Arab Conquest.

You may be interested in the book At Empire's Edge: Exploring Rome's Egyptian Frontier by Robert B. Jackson