It is also possible that the quote refers to the general doubt of the truth of his documented travels. If there is more evidence to support this interpretation, then I would like to ask what aspects of his travels did people doubt during his time?
Jacopo da Acqui's report of Marco Polo's last testament should be put in its proper context. His family members were trying to get him to repent and disown all the lies he'd written, to which Polo sneered that he had not told even half of what he'd seen.
Assuming the anecdote is true (or even if it's an invention by Jacopo, defending his subject, which seems just as likely), there are two ways to take it. First, if Polo told the other half, it would be realistic enough to make the whole story seem plausible. Second, the stories in the other half were even wilder.
Fortunately, we have ways to investigate both possibilities. Boring one first.
Scribes Gonna Scribe
The manuscript tradition of the Travels is a mess. It's maybe not as bad as Piers Plowman, but it's a mess. We don't have the original. There's no single surviving manuscript from which all others derive. Which is to say, there is at least something missing from/added to every existing version.
Some changes are surely accidental, or the result of translation problems. Others, though, are substantial omissions/additions. In those cases, it seems likely that somewhere along the way, a scribe/translator either thought the text needed that story, or saw that story and thought it did not belong for whatever reason.
One good example is the large void in the overall narrative, which occurs when Polo is basically hanging out in the East. Some manuscripts don't really say anything. Others explain that he was specially chosen as governor of a city for three years. Oh, and that he, his father, and his uncle pretty much single-handedly won a siege for the Khan by reinventing the trebuchet.
A passage like this one, especially since it was almost certainly added to some rather than omitted from the rest, suggests a couple of possibilities for reconciling skepticism/maintaining the book's veneer of "plausibility," fully aware it was just a veneer and part of the genre.
First, it fills in a large temporal gap at least in part. Useful in and of itself. Second, it casts the Europeans in a really good light. It's usually thought that even when pointing out good qualities of non-Christians, medieval European travel narratives relate tales and descriptions in ways that emphasize their Otherness. Massaging the awesomeness of the Polos serves those ends quite well--especially useful in a post-fall of Acre world.
A lot of additions seem to point to scribes feeling that the version they had in front of them was just missing a few details. Like a description for how horse thieves or bar brawlers were punished will have the procedure for punishing murderers added to it in later recensions. If the scribes thought the story needed it, Polo's friends and family could well have thought the same.
But really what you're here for is shipwrecks and cannibals, right?
Diamond Poop, or, 1001 Mediterranean Nights
In this earlier answer, I discuss how one episode that Marco Polo recounts traces back in time to amazing 12th century Persian poet Nizami, to a 10th century natural history text in Arabic, to a 4th century Christian bishop, to...Herodotus. From Polo:
Among these mountains there are certain great and deep valleys, to the bottom of which there is no access. Wherefore the men who go in search of the diamonds take with them pieces of flesh, as lean as they can get, and these they cast into the bottom of a valley...When the eagles [who also eat the deadliest snakes known to the ENTIRE WORLD] see the meat thrown down they pounce upon it and carry it up to some rocky hill-top...
The people go to the nests of those white eagles, of which there are many, and in their droppings they find plenty of diamonds which the birds have swallowed in devouring the meat that was cast into the valleys.
The details of the story change (Polo's diamonds are Herodotus's cinnamon sticks), but the underlying "plot" is the same.
This matters for present purposes because it shows how stories like this one are circulating around the Mediterranean-Asian world, crossing geographic and linguistic barriers as though they didn't exist. Ibn Battuta, Marco Polo, any interested traveler might well have heard the same basic story, with different details (snakes? no snakes? Alexander the Great? Random peasants?) from multiple sources. Heck, the valley of the diamonds story will even eventually be recorded in 1001 Nights (the messiest manuscript tradition yet), although the MS is more recent than Polo but the story was probably part of the collection beforehand.
The Mediterranean-Asian world was a world of stories. So in addition to whatever Polo did/did not see with his eyes firsthand, there's what he "saw" in the sense of having heard or overheard. And thus, the question becomes: what stories or what kinds of stories might we expect, that Polo nevertheless leaves out?
Looking at a 10th century Arabic text known as the Marvels of India, I'm going to suggest that what he left out includes shipwrecks and cannibals.
Marvels of India, like Polo and Ibn Battuta, is a collection of anecdotes, although it doesn't really attempt to be a cohesive travel narrative of any sort. The reason I think it's particularly useful for present purposes is that taken as a whole, the book is really, really repetitive. How many stories do you need that emphasize THIS FISH IS REALLY BIG? (At least four in a row, at one point, to say nothing of elsewhere in the text). It's a mishmash of tales, of the "1001 Mediterranean Nights," just like Marco Polo. (And yes, it includes the 'valley of the diamonds' legend.)
And some of the most common themes suicide, monkeys, snakes, REALLY BIG FISH...shipwrecks and cannibals.
Mind you, cannibals and shipwrecks both appear in Marco Polo's books. But not like this. The cannibals of Marvels sometimes have tails. Sometimes the book's author goes into way too much detail about how the cannibals cook their meals. And over and over, we read the trope--that survives to day--of the shipwrecked sailors on the island of the cannibal king.
In a world of travelers, merchants, and educated people across cultures (or the product of one author with a really thorough education), Marco Polo would have heard a lot of stories from a lot of people. One way or another, some of them wound up in his Travels. And one way or another, Jacopo da Acqui had his character Polo defend the "truth" in his own voice--whether that truth was what Polo saw--or what he had heard from someone who had heard it from someone.