Unfortunately, the sources don't have that much more to say than "There was teaching at Oxford around 1100, and then at some later points."

The universitas or guild consolidates when a group of students (usually Italy) or teachers (northern Europe) banded together and sought corporate legal privileges, which seems to have happened in Paris, Bologna, and Oxford sometime around/a little before 1200. Before that, "higher education" for potential bureaucrats and clerics alike meant a group of students flocking to be instructed/lectured to by a famous master. Usually the support for this would come through locating near a famous monastery or based at a cathedral church--including the chance to pick up some extra money instructing monks, for example. So that is what we are dealing with at Oxford before the 13th century: individual masters gathering students around themselves.

That "earliest evidence" of teaching at Oxford is frustratingly vague. A series of letters by the French/Norman canon Theobald of Etampes (outside Paris), dated to around 1100, has two where he describes himself as "magister Oxenfordiae." That is, master at Oxford. It's mentioned elsewhere that he lectured to both secular clerics and to monks, but there's not much to be said about individual tutoring or in-depth programs of study. The next reference to an Oxford master is Robert Pullen in the 1120s, from the Oseney Abbey Chronicle.

Oxford doesn't necessarily fit what we might consider the "ideal" for a potential high medieval university town. It didn't have the cathedral church--far from it! Oxford wasn't a diocesan seat until the 16th century; that was Lincoln, and Lincoln and Oxford were...kinda rivals at the time, in fact. And while there were plenty of monasteries--the earliest references to settlements at "Oxneford" are convents--the controversialist letters of Theobald as well as later mentions of lecturing to diverse audiences and (apparently) the eventual legal statutes of Oxford University suggest that the early masters were not basing themselves out of a convent.

What Oxford did have going for it was location: it's very conveniently situated on the Thames, near but not in London. It was definitely an up-and-coming city by the time Theobald arrived: a castle built just to its north became a stop on the English royal iter; both Richard I and John were born there during the king and queen's travels.

It's possible that the lack of domination by either a great monastery or a diocesan cathedral was a financial benefit for Oxford's masters. In addition to Theobald, mentioned as lecturing to regular and secular clergy alike, late in the 12th century Gerald of Wales makes a similar claim for himself. (He also mentions lecturing to lay people, which is fascinating). But this has to remain speculation.