I could answer your question and cite this as a source but it's not very long so I expect you'll just want to read it if you are interested in this topic: https://muse.jhu.edu/journals/shofar/v024/24.3schoenfeld.pdf

Edit: Ok, I will do my best to summarize. Jews did serve in the roman military in various capacities from early in Augustus' rule to the 5th century CE. Jews in the Roman military came from many different places. Some were settlers in Rome or other parts of the Italian peninsula but most came from Roman provinces such as Egypt and Armenia. Some Jews volunteered for the Roman army to gain citizenship or increase status and others were conscripted. Jews served many roles in the military and could even gain high rank. A notable example of this was the Egyptian Jew Tiberius Julius Alexander (http://en.wikipedia.org/wiki/Tiberius_Julius_Alexander) who may have attained the rank of Praetorian Prefect and was second in command to the future Emperor Titus during the Siege of Jerusalem. There is some debate as to how "Jewish" these soldiers are but the article indicates that many of these soldiers maintained some kind of Jewish identity.