Octavian was adopted by Caesar in his will, hence he legally became Caesar's son and was held to inherit by virtue of this fact. The adoption was not without some opposition, however, as Antony held it the will to be a forgery--largely because of his rather unexpectedly minor place in it.

would the Egyptian people have accepted Caesarion as their rightful king, despite his bastard status?

I'm a little confused here. Caesarion did become king of Egypt, sharing the throne with his mother. As part of the Donations of Alexandria Caesarion was also declared Caesar's legitimate heir, in defiance of Octavian's continued existence. This, however, was purely a political declaration and could have had no legal impact, as by Roman law Caesarion, a non-citizen and foreigner (worse yet a foreign king), could not legally inherit. If you're asking about whether he could have been recognized as Caesar's heir according to Roman law, the answer is a resounding "no." By Roman law the only way non-citizens could inherit was if they were the slaves of the testator, and only if they were freed in the same will that inherited them. During Caesar's lifetime citizen women couldn't even inherit from the first centuriate class, according to the stipulations of the Lex Voconia.

There are two separate legal principles going on here, completely unrelated to each other. The situation becomes somewhat clearer when it's pointed out, as I have already, that Octavian inherited by virtue of adoption--his distant blood relation to Caesar had nothing to do with anything