Because Greek, Latin, historical transformations.

All these biblical names did not enter English directly from modern transliteration practices (where you will often find them written with a y). They went through Greek and Latin, or sometimes just Latin, first.

Greek represented all these words with an iota. so, Ἱησοῦς (Iēsous, Ἰεροσόλυμα (Ierosoluma), Ἰωανάν or more commonly Ἰωάννης (Iōannēs).

Latin, similarly, used i to represent both a consonantal and a vowel sound. That's why, again, Iulius in Latin, but Julius in English. So Latin, taking over the Greek renderings of the Hebrew, pretty much had to render them in the same way: Iesus , Ierusalem (but, more commonly, Hierosolyma, because the Greek version was associated with the Greek word ἱερος which started with an h sound), and Ioannes.

The distinction between j and i is normally traced back to Gian Giorgio Trissino in 1524, and late modern Latin typically uses j for consonantal uses, i for vocalic ones. But this distinction also saw j come into more common usage in other languages, including English.

English had already been using i for both that consonantal i sound (aka, yod) but also the affricative /dʒ/ , but from the 17th century started to use j to represent the latter. When reading texts with j in them, including Jesus et cetera, J is read with typical English orthographic rules as /dʒ/, not as a yod-type sound.