My own googling seems to give conflicting accounts. Specifically around the time of Julius Caesar, some sources seem to say that as a province it was governed by a prefect (Pilate) and some say it was a semi-autonomous kingdom ruled by a king (Herod.) How exactly did this system work?
I think you are getting confused about chronology, as the system changed over time.
Herod the Great
Rules as a client-king after he overthrow Antigonus in 37 BC. Basically Rome employed client-states a great deal, particularly in the East, for a number of reasons. Firstly, people tended to be less rebellious if part of their political structure and national identity was left in tact. Secondly, it removed some of the administrative burden from Rome. Thirdly, it sometimes functioned to provide a buffer against other border states, i.e. the Parthians.
Make no mistake though, being a client-kingdom was very much a dependent existence. The crown was received from Rome, and answered to Rome. Herod had to show political capability and personal allegience to his Roman patron(s). Particularly he had to move quickly to switch support from Mark Antony to Augustus when the latter came to power.
Herod died in 4 BC, and his kingdom was carved up into three pieces.
Herod Archelaus
One of Herod's sons, Archelaus, took over from Herod as ethnarch of Judea (as well as Samaria and Idumia). Josephus writes about him in Antiquitis 17.8.4 and following, as well as Wars 2.1.
His title had to be granted by Augustus, and while it's not entirely clear, opposition to him may be some of the reason he did not carry the title king. overall he was not well liked or very competent, especially with a reputation for cruelty and flagrant violation of Mosaic law. So in AD 6 the Romans deposed and exiled him.
It's at this point that Judea turned into a Roman province. It was ruled by Prefects until AD 41, and then procurators from AD 44 onwards. Not all of these are known by name. Technically it was neither a senatorial nor imperial province, but a kind of special administrative region, dependent upon Syria, that is why it's prefects tended to be equites, rather than senators.
It's in this period that Pilate reigns as Prefect, from AD 26-36.
Between AD 41-44 Judea was again a kingdom, as Claudius made Herod Agrippa to be King of hte Jews, this was part of Claudius' more widespread policy to elevate procurators to governorship. Things get a little messy from the AD 60s onwards, with the revolt, the presence of a Roman legate; from 135 onwards the area was reorganized, and the province of Syria Palaestina emerges.