The main tactical reason for high losses to KV-1s and T-34s is simply that, during WW2, tanks were not the only thing that fought tanks. While a PzII, III or IV might struggle to destroy a KV-1 or T-34, AT guns, artillery or infantry could all be used to destroy it. The German 88mm AA gun was commonly pressed into an AT role, and was highly successful at it. Even if the smaller German AT guns had problems penetrating the armour of a Soviet tank, they could still imobilise it, so infantry could destroy it with grenades or demolition charges. Very few tanks of the time had armour sufficient to protect against large calibre HE shells, as fired from artillery pieces. Even if such a shell didn't penetrate, it would often stun the crew, or shock them into abandoning their tank. In addition, during the early stages of the war, Soviet armoured formations were large and unwieldy, with limited supporting arms - they had hundreds of tanks, but very little infantry or artillery. German units were more balanced, meaning that they were better able to take advantage of these alternate methods of destroying tanks, while the Soviets had very little with which to respond.

You also have to consider the wider strategic picture. During the opening stages of the Eastern Front, German advances cut off large Soviet formations. While a T-34 or KV-1 was more than a match for any German tank of the time, that assumed they had fuel and ammunition. By encircling much of the pre-war Soviet army, the Germans were able to inflict major losses, without actually having to fight them at full effectiveness.

Finally, part of the discrepancy may result from different statistical methods. Put simply, the Soviets counted tanks that were damaged, but repairable as losses, while the Germans counted only those that were both destroyed and irrecoverable as losses. This means that, even for a period where the two sides never saw combat, the Soviets would record several losses due to engineering problems, even if those tanks were later repaired. See here for a bit more on this.

At the start of the war, the vast majority of Soviet tanks were not KV or T-34s. There were maybe somewhat over one thousand of them combined on the Eastern Front out of more than 20,000 tanks.

The second reason is that people generally overstate how good those tanks were. Early versions of them were quite unreliable and would breakdown frequently. That by itself wouldn't be a huge deal since they could be repaired - however during a retreat there is very little time. It was not uncommon for Soviet crews to destroy their own tanks because they could not be repaired in time or they lacked fuel - just so that they wouldn't fall into German hands.

The last reason is that tactics matter at least as much as the tank itself. You can do very well with an inferior tank if you use it properly, and you can do poorly and lose many good tanks if you use them improperly. In the case of the Soviets, at the start of the war they still had only a vague idea of how to use them. They organized tanks into giant corps with several hundred tanks each but gave them very few supporting infantry or artillery compared to analogous German formations. So usually the tanks went into battle without proper support. Also, according to the mobilization plan, the Red Army was supposed to requisition trucks and equipment from civilian use to help with the logistics of moving infantry and artillery. However because mobilization was not complete at the start of the war, tank units often lacked proper logistical support. Finally, Soviet commanders in charge were often not making the best decisions, to put in mildly. Junior officers on the ground, too, lacked experienced with armored warfare. All of this put the Soviet armored forces at a severe disadvantage relative to the Germans and led to horrible losses. Within about two month from the start of the German invasion, the Soviets lost the vast majority of their tanks.

During the second half of the war, the Soviets were able to make much better use of their tanks. However by then the German armor had improved as well. The T-34s were no match for Tigers, Panthers, as well was the higher caliber anti-tank guns the Germans had put in the field by then.