A dive bomber by definition can carry bombs. Is there something else you're thinking of?

The immediate example that contradicts the frontal offensive armament is probably the Ju-88 non-night fighter series and arguably the He-177 (though it was medium angle dive bombing as opposed to steep angle, it was apparently capable of doing so) didn't have any frontal armament. Also I'm fairly certain some variants of Do-217s were dive bomb capable and had no frontal offensive armament.

So the short answer is "no."

As far as I remember and what I can find, the non-offensively armed dive bombers (though the definition of what was and was not a dive bomber differs depending on people) probably would be the Ju-88, He-177 and Do-217.

The most I could find on the Junkers JU87D (which was used in the conquest of Poland and France, but was retired in the Battle of Britain) in terms of Armaments is this:

Two fixed forward-firing 7.9-mm MG-17 machine guns or two 20-mm MG 151 cannons and a twin 7.9-mm MG81z machine gun in rear cockpit; one 3968-lb bomb (short range) or one 551-lb bomb under fuselage and four 110-lb bombs under wing (a wide variety of loads could be carried)

So basically there was a Pilot and a rear gunner inside the cockpit which was on the top of the plane. the rear gunner protected the plane from enemy aircraft coming from the rear, and the pilot had control of the bomb and from weapons.