In Germania by Tacitus and works by Pliny the Elder, both mention the great size, strength, and bravery in battle of the Germans. They also mention their powerful blue-eyed gazes and tawny hair. Many believed that the Germans were the most formidable of Rome's enemies. The Graeco-Roman ethnographic tradition was deeply entrenched in the concept that geography determined physical, mental, and cultural properties. The arid weather in Africa meant that Carthaginians and Egyptians were smaller and dark skinned but were very intelligent and capable of great engineering and battle tactics, while the Scythians, Celts, and Germans were large, fair-skinned, and not very intelligent because of the lack of food and cold, harsh weather. For this reason, some Romans, like Tacitus, writing after the fact, did not fear an invasion by the Germans as they did not believe them intelligent or unified enough to mount a well-supplied invasion and beat the Romans in the open field of battle. As others have pointed out in this thread, this is quite untrue, from the Cimbrian War, the Batavian Uprising, the Great Illyrian Rebellion, and the invasion of Italy during the Marcomannic Wars (see the excellent post by /u/PapiriusCursor below for more information), as well as countless skirmishes where the Germanic peoples showed their military prowess over the Romans.

One of the greatest military blunders against the Germans was the Battle of the Teutoburg Forest, where Varus was betrayed and tricked by the Germanic Tribes under Arminius, a Cherusci prince raised in Rome, and wiped out by a series of guerrilla ambushes. This was the straw that broke the camels back after a series of wars, and no great pushes into Germania were made afterwards. The greatest extent into Germania by Rome was a bit east of the Rhine and north of the Danube, where there was a buffer region and a large amount of trade for Germanic gold, furs, and amber.

What is dubious about many of the reports on the appearance of the Germanic peoples and attempts to categorize them as separate from the Celts, is that the same stereotypes that I mentioned above regarding the Germans are the exact same stereotypes attributed to the Gauls before they were conquered and pacified. They were seen as wild, suicidally brave, strong, fair-haired, and not as intelligent as the mediterraneans. The primitive, nomadic culture and noble aspects of Germanic society (the Romans glorified their rustic past and many historians and intellectuals were disgusted by contemporary Rome's greedy, lustful, and petty ways) were also the same that were attributed to the Scythians.

So there is definitely some truth to reports of the Germans, but take them with a large grain of salt because much of what was written was written by authors who had never met a native German and were basing their works off of reports and stories and was heavily distorted by stereotypes so that they could match their ideas and views of the world.

As to the Britons or the Picts, I have not read much on them so I would not presume to speak with credibility on the topic, but I am sure someone will have interesting information about them. All I know is that Rome never really expanded beyond Hadrian's wall, and the Picts only disappeared after the Viking Invasion and the Pictish identity was slowly forgotten and replaced by the that of the Gaelic Scots. Great question, I love this topic!

EDIT: To clarify, the Carthaginians technically were not North African, but Phoenician, and thus a Semitic people. However, the ethnographic theory of southern peoples extended to the Levant as well, and the military genius of the Carthaginians was definitely enough for the Romans to want to attach a geographical reason for their dramatic successes over Rome.

EDIT II: /u/Chie_Satonaka and /u/PapiriusCursor have pointed out an error I made regarding the military history between the Romans and the Germanic peoples, which was much more complicated than I led on. I was not aware of some of the wars that they brought up and I thank them for educating me. I have altered my post in light of this.

Germanic tribesmen and Celts do indeed appear to have had a deserved reputation for size and strength. /u/jebei gave an informative answer on this issue a while ago: https://www.reddit.com/r/AskHistorians/comments/2qexcq/how_big_was_the_average_viking_warrior_how_would/

Essentially the descendants of "Vikings", warriors of Germanic and Scandinavian peoples, probably the larger and stronger members of their societies due to their chosen occupation (going into often well defended lands and smashing them up), were around 173+ cm tall on average in 900-1200, which is about 5 foot 8-9.

Due to its good citation of sources, I'm going to do something unusual here and link an answer on a website (a bit like this one) for comparative Roman heights: http://history.stackexchange.com/questions/17072/average-height-of-ancient-roman-men-and-women

Here it is argued that the average Roman man appeared to be in the high 160s cm tall on average, or five and a half feet. Note that Vegetius, writing on selecting recruits for the Roman Army in the Late Empire, suggested heights of 172-177 cm or 5 foot 7 to 5 foot 9 (Vegetius Book 1.5) for elite infantry units or cavalry. However, Vegetius then proceeds to explain that as they really need more men, the height requirements are less important than their physical robustness (1.6). Roman concern for relative height between them and Celts and Germans, combined with a disdain for a lack of discipline and organisation in the latter which makes them ineffective soldiers despite their size, shows up often as a topos the literary sources. /u/t1taniumexp0se outlays some of these cliches. One that sticks in my mind is Polybius' analysis of Celts, who while he considered them to be very brave and formidable (2.29-30), they lacked robustness and endurance (2.78), which were classic Roman physical qualities that allowed them to be successful against larger enemies (combined with superior equipment and tactics, as Polybius mentions in 2.30).

There are a couple of minor points in the quite good post of /u/t1taniumexp0se that I would like to talk on while I'm here. The first is this:

the only time the Germans led an invasion like this before the Goths at the end of the Eastern Empire was the Cimbrian war in the late republic, whose forces were crushed by Gaius Marius

Alas, this is not true. The Romans had a great deal of respect for the fighting power of trans-Danubian and Rhine warrior peoples, and they had many scares throughout their history. Teutoberg Forest (which didn't actually happen in Teutoberg Forest but that's an issue for another day) was the most prominent incident since Arausio (which Caesar recalled to explain his attack of the Helvetii in 58 BC, kicking off the Gallic War), but between AD 9 when Varus lost his legions and AD 378 when Adrianople occured, countless major wars including serious reversals occured between the Romans and Germanic peoples. The Great Illyrian Revolt is one, where Augustus was deeply concerned the king Maroboduus, who allegedly possessed 200,000 infantry and 9,000 cavalry, would conduct a general invasion of Roman territory, prompting him to send Tiberius with a huge army to take care of it (Velleius 9-10), which took years. And it was only at the end of this that the so-called "Teutoberg Forest" catastrophe occurred, forcing the Empire back into a state of emergency (Velleius 120). Off the top of my head, from the 160s AD, the Empire under Marcus Aurelius had huge issues with the aggression of the Marcomanni and the Quadi, who even managed to attack Italy, destroy or siege major cities in the Roman heartland, defeated the imperial army in 170, and trashed Greece (Rankov and Austin, Exploratio, 205-10; see the Marcomannic Wars). These military crises from every more numerous and organised Germanic tribes and confederations continued to press the Empire for the rest of its history.

Also:

The greatest military blunder against the Germans was the Battle of the Teutoburg Forest

Depending on what you call a German, that honour could be reserved for Adrianople. You could also say the defeat of Marcus Aurelius' offensive in 170 AD was also probably far more worrying to his power and personal safety than was the defeat of a subordinate under Augustus: such reversals tended to get Emperors assassinated. Just personally I would also question whether a defeat automatically constitutes a "blunder".

I've gotten off topic here a little. I can't speak for the Picts or Anglo-Saxons in particular, but I am quite confident in saying that the endless Roman literary fascination with the physical size of Celts and Germans, and the corresponding warm glow of satisfaction that it brings a Roman reader in knowing that the legions conquered these people, was based in some reality. Of course, there were no shortage of very big Roman soldiers too, especially as they began recruiting heavily from Germanic peoples on the Rhine and Danube frontiers from as early as the first century AD. As Vegetius says, the minimum standard for an elite Roman infantrymen or cavalrymen would look about right these days, and some must have been larger. But on average, the Germans were big and strong and pretty scary, and once they were numerous, technically advanced and organisationally sophisticated enough, they were able to capitalise on the Roman Empire's misfortunes and replace it totally.