Throughout the Medieval and Early Modern periods, scholars who did not share a common language would often write letters to each other in Latin. When scholars visited each other, would they also speak in Latin, or was Latin a purely literary language by this stage?

For the record: Latin was spoken in Catholic Mass until the 1960s. Most Latin instruction, where it's available, requires speaking the language, even today.

So the answer is a bit simple: if two such scholars met, they would essentially be forced to speak in Latin due to the lack of shared language. There would likely be some regional differences in pronouncing the Latin, but the two scholars would be able to speak easily on just about any topic. Although this system began to decay due to the rise of vernacular (particularly in Protestant areas) throughout the early modern period, scholarly works were still typically printed in Latin—Newton's Principia, published in 1687, is entirely in Latin and was reprinted in Latin under Newton's direction for two eighteenth century editions. Even when scholarly works were printed in vernacular, they still often (and still do, to a lesser extent) drew heavily on Latin words and phrases because the entire history of Western discourse was written in that language.