I'm curious, what lightly armored opponents do you mean?

Do you have an example for reference?

I'm not entirely sure where you get the idea that it'd be terribly cumbersome. Aside from the fact that you're talking about a fairly broad span of time most of what would typically constitute a common idea of samurai armor isn't any worse for mobility- as a point of design- than comparable European armor.

What era are we talking about?

The early samurai (late Heian, early Kamakura) armor (like that linked by /u/cnzmur) could afford to be boxy and bulky because they were horse archers fighting horse archers. The people fighting on foot, if they were mentioned at all, had a tendency to just lose badly. The samurai's horse did the running and maneuvering. Their armor just need to let them shoot and protect against arrows. Though bulky and heavy, the coverage we'd probably consider quite sparse as only vulnerable points need armour, the saddle provided some protection for the legs, and the samurai would still need to be able to draw his bow on horseback.

In the Sengoku the armor that developed was more close-fitting, gave more coverage, and often incorporated mail and/or breastplates. Since they were no longer boxy-bulky and from what I can find seem to actually be lighter than early armors, someone wearing it on foot should have no problem moving around. At least the reenactors had no problem.

While I am not aware of any literary sources that says what happened when a fully armored warrior fought 1 on 1 vs someone with just a haramaki or the like, the fact that they kept trying to increase the protection on the armor throughout the Sengoku and tried to mass-produce it for everyone, are in my opinion evidence enough that they were effective. At the very least, if you had the latest armour in the late Sengoku, you could probably shrug off gunshot at 50m+. If you had light, or out-dated armour, you'd be in big trouble.