Was it because of language? Weather conditions?
The Romans were mostly interested in conquering areas that had a high degree of self sufficiency which could provide a tax base for them to extract from. Poland and eastern europe (as well as parts of Germany) did not have the higher level of urbanism at this time as in comparison with some Celtic Gallic settlements, which were in many ways already integrated into the Roman trade network in the case of southern Gaul.
The Teutoberg Forest story is noteworthy in 19th century germanic interpretations as to why the "march of the Roman Empire" was halted, but in reality incursions into Germany continued even after that disaster, and they were intended less for conquest or expansion than they were to play "whack-a-mole" with barbarian tribes in a bid to force them into some kind of political structure that would be compliant with Roman diplomatic efforts.
It was basically a cost/benefit analysis that decided the prestige to be gained by conquering more territory was outweighed by the lack of financial benefits accorded to conquest.
This is the same reason why the Roman Empire had no desire to conquer the empty wastelands of the Sahara south of the Mediterranean coastlines.
Keep in mind too, our modern "nationalistic" desire for territorial conquest tends to stem from our thinking of geography in terms of maps. "Why is this empty area not ours? Lets go get it." The Romans thought in terms of where there was "civilization" (derived from civitates, which means city/urban/administrative area).
Sources: