The Pacific Theater isn't my field of concentration but I was curious about this question and did some basic digging. While we wait for potential answers (from someone more qualified), I thought it was worthwhile to share some relevant information.

From a 2015 JiJi Press article, here is a primary account retelling from Kiyokazu Tsuchida of the IJN:

カニとかなんとかは生で食ったりとか。食料はアメリカ兵から。「親方日の丸だい」って日本で言うでしょ。「親方星丸だい」って言って。泥棒して国のためになるんだからこんないい仕事があるものか、って。みんなもう。（戦後に残存兵が洞窟から）出てくるときでも、３年間分くらいの食料は確保していたですから。

私はね、（終戦後に）書き置きして結局ほら、あれ（脱走）したわけです。皆（日本の敗戦を）信用しなかったですね。ところが敵のごみためを漁りに行くと、東条英機があれしたとか、沖縄にアメリカ兵が立ったとかいろいろなことが。それでもね、なんとか工作でこうやって載せているわけで、だまされるなっていうような人がほとんどだったですよね。日本は負けてないって。機動部隊のあれ（反撃）を待っていたわけですよ、我々は。

Here is a rough translation, corrections are welcome:

Crabs or something of the like were eaten raw among other things. Food was from the Americans. I guess as we say in Japan "Oyakatahinomaru" (this is a bit of a complicated term, it means "Hinomaru is the master". Hinomaru as in Japan the country and the central government.) "Oyakatahoshimaru" as it's said. I wonder, to be a thief for the sake of the country, was this really good work? Everyone else thought the same way. (Other cave-dwelling holdout soliders) Even when we came out of the cave, we had enough food to last us 3 years.

You know for me, (after the war) I wrote things down you see, the escape. Everyone didn't believe in it (Japan's defeat in the war). However, as I went rummaging for the enemy's garbage, I found various things [on newspapers or such] like Hideki Tojo or American soldiers on Okinawa. But still, you know, with this kind of deceiving handicraft that was put together somehow, most of us weren't fooled. Japan didn't lose. We were waiting for the Kido Butai to counterattack.

Tsuchida goes on to talk about his suspicions of the war's end from the news pieces he gathered. He thought perhaps Tojo was undergoing some medical examination. He observed B24's flying towards the Philippines, and he began to wonder if Japan was losing the war and they would soon be ordered to engage in the Philippines, but he kept himself quiet.

A similar second-hand account on Ei Yamaguchi reinforces the scavenging efforts:

Yamaguchi and his band of holdouts subsided mostly on stolen US goods, including weapons. It seems as though the stragglers favored the M-1 carbine because of its light weight. Many improvised and cut off the end of the barrel making the gun more mobile and light. When I explored Yamaguchi's cave, it was amazing that all the artifacts were US Army/Marine equipment. We found US water cans, pineapple grenades and helmets. Yamaguchi chuckled and told me they "liberated" the stuff from US dumps. He also related his experience of sneaking into the American lines and watching movies on many occasions!

A brief mention of cave holdouts are mentioned in "Peleliu 1944: The Archaeology of a South Pacific D-Day." by Neil Price and Rick Knecht in the Journal of Conflict Archaeology 7, no. 1 (2012): 5-48. With the end of formal combat operations in Peleliu in November 1944, isolated groups remained as resistance for a brief time. Suicidal attacks were made on occasion until most holdouts surrendered by February of 1945. Of course, the 34 person group emerged in April 1947, most of them were truck drivers led by the aforementioned Lieutenant Yamaguchi Ei. They are described as preparing for an attack on a lightly guarded airfield, and believed they could hold it until relief came from other forces. Notably for the island, one last holdout was a Korean slave labourer who was found stealing vegetables as late as 1954. The article focuses on other aspects of the battle on the island and no further information is provided regarding survival methods for holdout groups.

I was searching for more scholarly writings on the matter but my resources are limited. At the very least, we can see that many of the materials and food was scavenged from the Americans. While this addresses part of the question, a more detailed and proper answer is welcome.