I have some small knowledge of the feudal system, and when looking at maps (especially those made towards the end the HRE's existence) that most of the land was ruled by outside powers, while only some of the counties/duchies/kingdoms were, I assume, vassals of the HRE, while still being held by Prussia and Austria, which were not vassals of the HRE from my knowledge.
Well, the problem with this question is the nebulous nature of "empire". What exactly does that term mean? According to Google, an empire is "an extensive group of states or countries under a single supreme authority, formerly especially an emperor or empress." So under that definition, the HRE certainly qualified, because it has a pretty large number of member states that were all nominally subservient to a single central authority. But then, so would any federal system, but we don't call the US or Russia an empire. Rome didn't grant a whole lot of powers to the provinces, but most would call them an empire at their height.
My point is that the term "empire" is a very unscientific term, mostly given to states who decided that wanted to be called an empire. In light of this, evaluating how much of an "empire" the HRE was is a strange, basically unanswerable question. There's no good was to disqualify them from the title, because you'll probably find an exception for whatever rule you decide on.
Like most answers covering anything as long-lived as the Holy Roman Empire, the answer is basically "depends". You mention Prussia and Austria, though, so I'm going to assume that you're asking more specifically about the early modern HRE, which puts your question more within my boundaries and also allows a simpler answer: no, the HRE was not much of an "empire". Explanation below.
The term "empire" deserves some discussion, first. Having a sitting Emperor or Empress helps, yes, but that's by no means a given due to colonial empires (like Britain before 1877). Being strong relative to your neighbors helps, but we still have the Dutch Empire. Realistically, an empire is any state that contemporaries and/or historians agreed to call an empire.
But let's try to get in and answer your questions directly. First, Prussia and Austria were both actually seated within the Empire and remained within it until its disbandment under Napoleon. The princes of Austria were in fact Holy Roman Emperors for essentially the entire early modern period, and wielded great influence throughout Europe; on their own, the external territories of Prussia and Austria do not actually suggest a weakness in the HRE. In fact, they actually strengthened the HRE's defense against foreign invaders.
However, the historical realities make it pretty clear that thinking of the HRE as an "empire", or rather a meaningful political unit, is misguided at best. The northern and southern halves of the HRE spent as much (more, if I'm not mistaken) warring against each other over religion and influence than against foreign threats, and the entire people of Germany suffered dearly as a result. Each individual princedom in the Empire, of which there were hundreds, effectively ran their own territory as they saw fit without much interference from the Emperor. Citizens with the Empire would have legal recourse to the Imperial justice system if they felt wronged, which did indeed save many Jews from blood libel trials and convictions, but generally struggled more against standing local customs and interest groups than against any higher power. This arrangement did vary, however, as the Reformation repeatedly led the Emperor into attempting to enforce religious unity and many territories to band together into military alliances (with the early Protestants even considered creating a new Empire of their own prior to their losses in the Schmalkaldic War).
So how should we actually describe the HRE? Wikipedia describes it with a rather awkward turn of phrase in order to say something vague enough to cover 900 years at once: "a multi-ethnic complex of territories in central Europe". For the early modern period, it's probably best to describe it as a loose confederation of numerous territories under a nominal emperor. It's essentially a unique historical circumstance, and even people of the time pretty openly recognized that attempting to really describe the HRE was a hopeless endeavor—but we can be certain that just calling it an "empire" is completely misleading.