You could argue it two ways. First, suppose (as all right-thinking people do) that the Americans are rebels against their rightful king; then the colonies are in fact English clay, and the Royal Navy has, presumably, every right to search vessels engaged in trade with dominions of the English Crown. They might be smuggling, they might have English subjects on board who are subject to conscription into the Navy, they might be bringing weapons to the rebels - in short, if you're going to enter an English port, whether or not the Crown currently controls it, you're subject to inspection.

Second, suppose (as the propagandists of "independence" and "democracy" would have you do) that the rebels are a sovereign nation. Then the Royal Navy still has the right to prevent you bringing them contraband, that is to say, cargoes of value for their war effort. In the 18th century this basically meant actual weapons; the doctrine that even food to feed the workers in the factories that made the weapons was contraband subject to seizure or sinking belongs to a much later era. The status of 'neutral' carries obligations as well as rights; you are obliged not to help either belligerent party, for example by allowing their troops through your territory, or by shipping them guns. The belligerents, on the other hand, have the right to make sure you are following your obligations.

Finally, consider that there's such a thing as a false flag. Sure, the ship is flying the orange-white-blue Prince's Flag; but dye is cheap. Even rebels might be able to afford some. An international law that tried to make a scrap of cloth be a protection against inspection would not be respected in time of war, since then the weaker belligerent would just fly neutral flags in all his ships. Wisely, the people writing international laws in the 18th century didn't put any such law on their books; they limited themselves to protecting non-military cargoes, which even a belligerent power could see its way to respecting.