I understand that Octavian invaded it in his war with Antony, but prior to that it was already some sort of client state, hence Antony staying there, when exactly did this happen?
I actually talked about how the relationship between Rome and Egypt evolved prior to Cleopatra meeting Julius Caesar here which goes into detail on the events immediately leading up to the situation you describe and goes back to the Punic Wars.
I want to clarify some points in your question that that thread does not cover however. Antony only moved to Alexandria around 34 BCE, as relations between him and Octavian broke down and this was due in large part to the division of the East by him and Cleopatra. Alexandria was never his official "HQ" or anything of the sort contrary to what shows like HBO's Rome portray, instead his decision to spend time in Alexandria was scandalous ammunition against him because it was a foreign capital in a way that Athens was not at the time. Egypt was a "friend and ally" of the Roman Republic at the time, and in truth the power balance between the two was heavily tipped in Rome's favour but it was not a part of Rome by any means. Even within Cleopatra's lifetime Roman military interventions like Aulus Gabinius' campaign to restore her father Ptolemy XII to the throne had to be justified through a host of legal loopholes, and Caesar's campaign to restore her was justified by her father's will which left the succession to her and her brother and which named the people of Rome as its guarantors. At no point in time was there ever a fixed Roman policy towards Egypt by the Senate or the various generals and politicians who rose to prominence, some wanted it conquered, others wanted to prevent this because it would upset the balance of power too much while others simply did not pay it much mind. Within Egypt, there was some hostility towards Roman influence as they were sometimes seen as an arrogant, damaging and dangerous foreign element in the country (a perception which individuals like Gaius Rabirius, Julius Caesar and even Clodius Pulcher after his conquest of Cyprus) did nothing to ease. At the same time, the potential that alliances with Rome had to offer in the way of military and political support was keenly felt by Ptolemaic rulers and even Ptolemy XII's scapegoating of Rabirius to deflect harsh feelings over increasing taxation and state debt made use of Egypt's Roman allies to serve the crown's interests.
I can give a bit of an overview of this, at least until someone more knowledgeable shows up: It was a fairly slow process that I believe started during Pompey's campaigns in the east. He gave support to Ptolemy, helping secure his control, so I suppose that's when it really started.
Rome was the big fish for a while up to that point, and while Egypt was rich, they weren't all that powerful, so they ended up paying bribes to prominent romans (like Caesar and Pompey) to keep the nile region off of Rome's dinner menu so to speak. Pompey turned The pharaoh Ptolemy into a client king, meaning Egypt was nominally independent but absolutely subservient to Rome diplomatically.
Years later during the Roman civil war, Pompey fled to Egypt to gather support after losing a major battle in Greece, the old king had died and his young son was fighting with his sister/wife cleopatra for dominance. The Young Ptolemy ended up having Pompey killed in what he thought would be a safe bet to curry favor with Caesar who was hot on Pompey's heels. This was a BIG mistake, Pompey and Caesars relationship was complicated, and they had been at war, but Caesar wanted to make peace with Pompey, not kill him. So when he arrived in Egypt expecting to be pointed in the direction Pompey had gone and was instead presented with his head, he was furious.
To make a long story short, Caesar chose to side with Cleopatra in the Egyptian civil war, and when he left she was the ruler, and Egypt was still nominally free from Roman rule, albeit with very close economic ties to Rome.
Later when Octavius came to end his war with Antony, he took personal control of the region. So that's when it lost its "nominal" independence. Egypt would remain the personal property of the emperors until basically the end of the empire.