Depending on the time period you refer to, what entails being a 'samurai' changes somewhat. In arguably the most (in)famous period of Japan, the Sengoku era, the answer is very definitely a no. But, similar to the knight in continental Europe (in concept, not as an analogous question), they were around for a long time, and depending on which time period you refer to, the answer changes.
You might be interested in one of my replies from many thousands of years ago which talks more about the changing dynamics of military doctrine throughout Japan (starting earlier than, and ending a little after, the aforementioned Sengoku era) than directly answering your question, but the answer can be mostly found there. In any case, given that the samurai (in quotation marks here) were around for a long time, as said above it depends - if you're referring to the precursors of the stereotypical samurai, then yes, they were the 'only soldiers' that Japan (which, as most of 'medieval' history goes, is a nebulous concept) had. On the other hand, leading up to and all throughout the Sengoku period, (conscripted) footsoldiers made up the bulk of the armies, and so in this case the answer to your question is (as mentioned at the beginning) no.
To add to what u/bigbluepanda wrote, both now and in the linked past, it also depends on how you define "soldier". Taking a middle-of-the-road definition as somebody who fights in an organised army, there were non-samurai soldiers in Japan. First, as already mentioned in the linked previous answer, there were the ashigaru, forming the bulk of Sengoku Period armies.
Second, there were other organised armed forces in Japan. The most important were:
Monastic armies. These, and the soldiers, were called sōhei, "monk-armies" or "monk-soldiers". By the Heian period, if not earlier, major Buddhist temples were maintaining standing armies to protect their territories, and to attack, or threaten to attack, rival temples, and to intervene in wider political disputes and wars. The rank-and-file soldiers might have officially has the status of monks (of the lowest monastic ranks), but their duties could be purely military. Other soldiers were officially lay people rather than monks (i.e., they hadn't taken monastic vows). Contemporary sources typically say very little about the size of monastic armies, but they could be large enough to deter daimyo from attacking. When Oda Nobunaga destroyed the temple complex on Mt Hiei, along with the monastic armies there, in 1571, he used a force of about 30,000 men. For a lot more on monastic armies, see Adolphson (2007).
Regional non-daimyo armies. The most famous, and perhaps the most effective, of these were the forces of Iga Province in the late 16th century. Iga sought to take advantage of its geography - mountains restricting routes by which armies could invade - to maintain its autonomy and avoid submitting to the warlords seeking to unify Japan. Iga of the time has been described as proto-deomcratic, controlled by a league, or ikki, of local leaders. Iga's army was powerful enough to defeat an invasion by a force of about 10,000 in 1579. Two years later, Oda Nobunaga deployed a much larger force, about 42,000 strong, and successfully conquered Iga.
Combining elements of both of these were the Ikkō-ikki, uprisings seeking regional independence or autonomy, linked to the Ikkō-shū of the Jōdo Shinshū Buddhist sect.
We can also add:
which appeared before the samurai disappeared.
References:
Adolphson, Mikael S.. The Teeth and Claws of the Buddha: Monastic Warriors and Sohei in Japanese History, University of Hawaii Press, 2007.